0% found this document useful (0 votes)

244 views

Benchmark Factory For Databases User Guide

Software described in this guide is furnished under a software license or nondisclosure agreement. No part of this guide may be reproduced or transmitted without the written permission of Quest Software, Inc. No license, express or implied, to any intellectual property right is granted by this document or in connection with the sale of Quest products.

Uploaded by

Christopher Alexander Castellares

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

244 views

Benchmark Factory For Databases User Guide

Uploaded by

Christopher Alexander Castellares

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 345

Benchmark Factory for Databases

User Guide

6.5

2010 Quest Software, Inc. ALL RIGHTS RESERVED. This guide contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchasers personal use without the written permission of Quest Software, Inc. The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document. If you have any questions regarding your potential use of this material, contact: Quest Software World Headquarters LEGAL Dept 5 Polaris Way Aliso Viejo, CA 92656 email: legal@quest.com Refer to our Web site (www.quest.com) for regional and international office information. Quest, Quest Software, the Quest Software logo, AccessManager, ActiveRoles, Aelita, Akonix, Benchmark Factory, Big Brother, BridgeAccess, BridgeAutoEscalate, BridgeSearch, BridgeTrak, BusinessInsight, ChangeAuditor, CI Discovery, Defender, DeployDirector, Desktop Authority, Directory Analyzer, Directory Troubleshooter, DS Analyzer, DS Expert, Foglight, GPOADmin, Help Desk Authority, Imceda, IntelliProfile, InTrust, Invirtus, iToken, JClass, JProbe, LeccoTech, LiteSpeed, LiveReorg, LogADmin, MessageStats, Monosphere, NBSpool, NetBase, NetControl, Npulse, NetPro, PassGo, PerformaSure, Point, Click, Done!, Quest vToolkit, Quest vWorkSpace, ReportADmin, RestoreADmin, ScriptLogic, SelfServiceADmin, SharePlex, Sitraka, SmartAlarm,

Benchmark Factory for Databases User Guide Copyright

Spotlight, SQL Navigator, SQL Watch, SQLab, Stat, StealthCollect, Storage Horizon, Tag and Follow, Toad, T.O.A.D., Toad World, vAutomator, vConverter, vEcoShell, VESI,vFoglight, vPackager, vRanger, vSpotlight, vStream, vToad, Vintela, Virtual DBA, VizionCore, Vizioncore vAutomation Suite, Vizioncore vEssentials, Vizioncore vWorkflow, WebDefender, Webthority, Xaffire, and XRT are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. For a complete list of Quest Softwares trademarks, please see http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered trademarks are property of their respective owners.

Benchmark Factory for Databases User Guide January 2011

Table of Contents
Getting Started Getting Started
Benchmark Factory Community

17 17
17

New For This Release

New for Benchmark Factory 6.5 New for Benchmark Factory 6.1.1 New for Benchmark Factory 6.1 New for Benchmark Factory 6.0

18
18 18 18 18

Understanding Benchmark Factory About Benchmark Factory The Benchmark Factory Environment
Integration with Other Quest Software Products

19 19 19
20

Understanding the Benchmark Factory Environment Understanding Benchmark Factory Terminology How Benchmark Factory Works Create a Script With Benchmark Factory Benchmark Factory Components BenchmarkFactory Console Agent(s) Run Reports Node
Cutting and Pasting of Run Reports Testing Server Throughput Testing SQL Scalability Testing Real Application Clusters

20 21 24 25 32 32 33 34
34 34 35 35

Repository Manager

Benchmark Factory for Databases User Guide Table of Contents

Views
Script View Jobs Queue View Properties View Agents View Output View Properties

38
38 39 40 40 40 41

How Do I... Overview How do I... User Scenario Wizard Profile Creation Wizard
Overview

71 71 71 72 72
72

Edit a Profile Submit Job Dialog Run Reports Node

Cutting and Pasting of Run Reports Testing Server Throughput Testing SQL Scalability Testing Real Application Clusters

74 74 76
77 77 78 78

Advanced Creation Objects (Object Mapping) Benchmark Objects Wizard Changing Scaling Factors
To Change Scale Factors

79 79 81
82

Execute File Wizard

Overview

83
83

BFScript Wizard

Benchmark Factory for Databases User Guide Table of Contents

Script Wizard Script Assist

84 85

SQL Scalability Wizard

Overview OverviewTab Transactions Summary Messages

86
86 87 88 90 90

Copy a Load Scenario to a Replay, Mix, Goal, or Scalability Test Running SQL Preview Replace Child Latencies Goal Testing Quick Start
Purpose of this Quick Start Actions Performed During This Quick Start

91 92 93 94
95 95

Use Global Search and Replace Creating an ODBC Trace File Import Profiles from the Repository Real Application Clustering (RAC) Quick Start
Overview Purpose of this Quick Start Actions Performed During This Quick Start

104 105 105 106

106 106 106

Save the Selected Job to the Script View Schedule A Job About Latency Set the Number of Users on an Agent Stagger User Startup Running Benchmark Factory with Multiple Agents

116 116 118 119 120 121

Benchmark Factory for Databases User Guide Table of Contents

Benchmark Factory Console Benchmark Factory Agent Running a Job With Multiple Agents

121 121 121

Creating Load Scenarios Load Scenario Wizard

Overview

123 123
123

Settings About Settings General Settings Benchmarks Settings Timing Settings User Load (Measurement Intervals) LatencySettings Repository Settings Agent Settings Drivers Driver Selection
Adding BFScript to the Profile Page IBM DB2CLI Microsoft SQL Server (Native) Microsoft SQL Server (ODBC) MySQL (ODBC) MySQL ODBC Oracle Sybase

125 125 125 127 128 129 131 133 134 136 136
136 137 138 138 139 139 139 140

Profiles Overview

142 142

Benchmark Factory for Databases User Guide Table of Contents

Edit Profiles Import Profiles Profile Properties

Overview General Tab Options Tab Profile Properties Real-Time Counters Tab Clustering Tab

142 143 143

143 144 145 146 149

Wizards Overview Benchmark Objects Wizard

Benchmark Objects Wizard

150 150 150

150

Data Repository Migration Wizard

152
152

Execute File Wizard

153
153

Load Scenario Wizard

Load Scenario Wizard Industry Standard Benchmark (TPC-C, TPC-H...) Validate Oracle RAC Cluster Scalability Test Database Scalability Test SQL For Scalability Replay Load from an Oracle Trace File Test Stored Procedures for Scalability Replay Load from SQL Server 2000/2005 Trace Table Import from Foglight Performance Analysis Repository Import from a Delimited Text File Replay Load From an ODBC Trace File

154
154 155 156 157 159 160 162 163 164 165 167

Benchmark Factory for Databases User Guide Table of Contents

Import from Oracle Dynamic Performance View Import from SQL Files Create a Custom Load Scenario (Scalability, Goal...) Mix Test Replay Test Goal Test Scalability Test

168 169 170 170 171 172 174

Profile Creation Wizard

175
175

BFScript
BFScript Wizard

176
176

SQL Scalability Wizard

179
179

Submit Job Dialog

184
184

Transaction Wizard
Transaction Wizard Test Stored Procedures for Scalability Create SQL Statement Transaction Create Prepared SQL Statement Transaction(s) Create Execute Prepared SQL Statement Transaction Import Transactions

186
186 187 188 189 193 193

User Scenario Wizard

User Scenario Wizard Import and Replay Load Scenarios From an Oracle Trace File Replay Load from a SQL Server Trace Table Import from a Delimited Text File Replay Load from an ODBC Trace File

194
194 194 195 196 196

Benchmark Factory for Databases User Guide Table of Contents

Create a Custom User Scenario

197

Testing Results (Run Reports) Benchmark Factory TestingResults Benchmark Factory Testing Results Terminology Exporting Test Results to Excel Using the Command Line Real-Time Load Testing Results
Overview-Viewing Real-time Statistics Workloads Change Graph Views Run Status Node Real-Time Statistics Node Summary Node Transaction Node

198 198 198 199 200

200 200 203 203 205 206

Benchmark Factory Run Reports

Run Reports Node Change Graph Views Run Reports Viewer View Load Testing Results Using Benchmark Factory Run Reports View Multiple Run Results Export Test Results to Excel

207
207 210 212 213 214 220 221

Benchmarks Overview
Realistic Expectations When Using Benchmarks What Benchmarks Measure Provided Benchmarks

222 222
222 222 222

AS3AP Benchmark
Best Practices

223
223

Scalable Hardware Benchmark

224

Benchmark Factory for Databases User Guide Table of Contents

How the Scalable Hardware Benchmark Works Scaling Factor Best Practices

224 224 224

TPC-B Benchmark
Overview Certification of Transaction Processing Council (TPC) Testing Results Best Practices Scaling Factor

225
225 225 225 226

TPC-C Benchmark
Overview TPC-C Tables Certification of Transaction Processing Council (TPC) Testing Results Best Practices

226
227 227 228 228

TPC-D Benchmark
Overview Certification of Transaction Processing Council (TPC) Testing Results Best Practices Scaling Factor

229
229 229 229 230

TPC-E Benchmark
Overview Certification of Transaction Processing Council (TPC) Testing Results Best Practices Scaling Factor

230
230 230 231 231

TPC-H Benchmark
Overview Certification of Transaction Processing Council Testing Results Best Practices Scaling Factor

231
231 232 232 232

Benchmark Factory for Databases User Guide Table of Contents

BFScripts About Scripts Using Scripts in Username and Password Fields Dates and Numbers
$BFCreditCardExp $BFCurrentDate BFCurrentDateTime $BFDate

234 234 236 236

236 236 237 238

File Access
$BFFileArray

239
239

Global Variable
$BFGetGlobalVar $BFSetGlobalVar $BFSetGlobalVarRtn

240
240 240 241

Name and Address

$BFAddress $BFAddress2 $BFCity $BFCompany $BFCountry $BFEmail $BFFirstName $BFFullName $BFLastName $BFMiddleInitial $BFPhone $BFState $BFZipCode

241
241 241 242 242 243 243 243 244 244 244 245 245 246

Benchmark Factory for Databases User Guide Table of Contents

Numerical Manipulation
$BFFormat $BFProd $BFSum

246
246 247 247

Random Numbers
$BFRand $BFRandRange $BFURandRange $BFURand

248
248 248 249 249

Random String
$BFCreditCard $BFRandList $BFRandMultiList $BFRandStr $BFURandList $BFList

250
250 250 251 252 253 254

String Manipulation
$BFAsc $BFChr $BFConcat $BFLeft $BFLen $BFLower $BFMid $BFRight $BFTrim $BFTrimLeft $BFTrimRight

255
255 255 255 256 256 257 257 258 258 258 259

Benchmark Factory for Databases User Guide Table of Contents

$BFUpper

259

Test Info
$BFGetVar $BFMaxNode $BFNode $BFNumberOfIterations $BFProfile $BFRunID $BFSetVar $BFSetVarRtn $BFUserCounter $BFUserID $BFUserLoad $BFNextUserload $BFPrevUserload

260
260 260 261 261 262 262 263 263 264 264 265 265 265

Quick Starts About the Quickstarts Real Application Clustering (RAC) Quick Start
Overview Purpose of this Quick Start Actions Performed During This Quick Start

267 267 267

267 268 268

Upgrading Your Database Server Quick Start

Overview Purpose of this Quick Start Actions Performed During this Quick Start

278
278 278 279

Oracle Trace Import-Database Replay Quick Start

Purpose of this Quick Start Actions Performed During This Quick Start

286
286 287

Benchmark Factory for Databases User Guide Table of Contents

Standard Benchmark Quick Start

Purpose of this Quick Start Actions Performed During this Quick Start

293
293 293

Goal Testing Quick Start

Purpose of this Quick Start Actions Performed During This Quick Start

300
300 301

Troubleshooting Agent Connection Use Benchmark Factory with SQL Server 2005 Client MySQL Initialization Settings Appendix Change Graph Views
Show Data/Show Graph Graph Legend Toolbar Print Copy Data to Clipboard Copy Graph to Clipboard Load Configuration Save Configuration Set as Default Clear Configuration

310 310 310 310 311 311

311 312 312 313 313 313 313 313 313 313

Customize List Controls Create a SQL 7 Trace Table Using the SQL Server Profiler Create a SQL 2000 Trace Table Using the SQL Server Profiler Create a SQL 2005 Trace Table Using the SQL Server Profiler Oracle Instant Client Installation

313 314 315 316 317

Benchmark Factory for Databases User Guide Table of Contents

Migrating Repository Data Using the DOS Command Line Store Procedure Examples Oracle Trace File Activation Support Bundle System Requirements/Upgrade Requirements/Supported Databases
System Requirements

317 318 320 321 322

322

Shortcut Keys Copyright, Licensing, and Licensing Agreement Licensing and Virtual User Packs
Modify the Benchmark Factory License Modify the Benchmark Factory Virtual User Pack

324 327 328

329 330

330 330 Appendix:Contact Quest Contact Quest/About Quest Software

About Quest Contacting Quest Software ContactingQuest Support

342 342
342 342 342

Index

343

1
Getting Started
Getting Started
Benchmark Factory is an intuitive, easy to use database load testing tool. The following links provide you with information to quickly start the testing process and gather results.
l l l l l l l l l

Video: Take a Tour of Benchmark Factory About Benchmark Factory (page 19) Understanding the Benchmark Factory Environment (page 20) Understanding Benchmark Factory Terminology (page 21) How Benchmark Factory Works (page 24) Benchmark Factory Components Create a Script With Benchmark Factory (page 25) Benchmark Factory TestingResults (page 198) Quick Starts
l l l l l

Real Application Clustering (RAC) Quick Start (page 267) Upgrading Your Database Server Quick Start (page 278) Oracle Trace Import-Database Replay Quick Start (page 286) Standard Benchmark Quick Start (page 293) Goal Testing Quick Start (page 300)

Quest Benchmark Factory Web Site (contains additional videos and white papers)

Benchmark Factory Community

Get the latest product information, find helpful resources, and join a discussion with the Benchmark Factory for Databases team and other community members. Join the Benchmark Factory for Databases community at http://www.benchmarkfactory.inside.quest.com.

Benchmark Factory for Databases User Guide Getting Started

New For This Release

New for Benchmark Factory 6.5
l

New Oracle Trace Record and Playback Wizard-A new wizard has been added that walks users through the entire process of capturing the Oracle workload, including the database backup. Enhanced Connection Profile Page-The connection profile pages have been enhanced to be more specific to the particular database vendor (Oracle, Microsoft SQL Server, etc.). Oracle Profile Improvements-The Oracle profile now supports multiple Oracle Homes, Direct Connections, as well as being able to specify "Connect As". The Oracle Profile also has a new Statistics page to allow the user to automatically capture Statspack or AWR snapshots during test execution. New Activation Window-The new Activation window provides trial users with more information such as expiration date, days remaining, licensing information, etc. New Licensing Window-A new licensing window is now available to provide better information about the user's licensing for the different Benchmark Factory licensing keys.

New for Benchmark Factory 6.1.1

This release of Benchmark Factory for Databases provides critical bug fixes.

New for Benchmark Factory 6.1

l l l

Support for 11gR2. New global variable support for BFScript. Enhanced bind types.

New for Benchmark Factory 6.0

The TPC-E benchmark is now available in this release for both Oracle and Microsoft SQL Server.

2
Understanding Benchmark Factory
About Benchmark Factory
Benchmark Factory is a database performance and code scalability testing tool that simulates users and transactions on the database and replays production workload in non-production environments. This enables developers, DBAs, and QA teams to validate that their databases will scale as user load increases, application changes are made, and platform changes are implemented. Benchmark Factory is available for Oracle, SQL Server, DB2, Sybase, MySQL, and other databases via ODBC connectivity. The following topics provides an overview of the Benchmark Factory Environment:
l l l l l

The Benchmark Factory Environment (page 19) Understanding the Benchmark Factory Environment (page 20) Understanding Benchmark Factory Terminology (page 21) How Benchmark Factory Works (page 24) Create a Script With Benchmark Factory (page 25)

The Benchmark Factory Environment

Benchmark Factory places enormous stress on a database system, which is typically hard to achieve in a standard testing environment. A system typically breaks under extreme load. By identifying system capacity and performance bottlenecks before they occur, Benchmark Factory facilitates proactive testing, which in turn reduces downtime, development costs, and potential loss of revenue. Benchmark Factory allows you to:
l l l l l

Determine system throughput and capacity for database systems Simulate thousands of concurrent users with a minimal amount of hardware Find applications that do not scale well with an increase in the number of users Find breaking points, weak links, or bottlenecks of a system Quantify application or server performance

All test results are collected and stored in the repository for data analysis and reporting. Benchmark Factory collects a vast amount of statistics, including overall server throughput (measured in transactions per second, bytes transferred, etc.) and detailed transaction statistics by

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

individual workstation producing a load. You use these statistics to measure, analyze, and predict the capacity of a system.

Integration with Other Quest Software Products

Benchmark Factory integrates with other Quest Software products to allow IT departments to quickly measure the capacity and performance of their systems to ensure that users will experience fast response times. For instance, Benchmark Factory can run multiple workloads in conjunction with Spotlight products. This provides the ability to detect and diagnose issues, allowing users to resolve bottlenecks, slow performance, and application flaws before an application is entered into production.

Understanding the Benchmark Factory Environment

Note: The Benchmark Factory Quick Starts provide procedures that introduce you to the Benchmark Factory environment. See "About the Quickstarts" (page 267) for more information. The following provides a narrative on the terminology, application, and mechanics of using Benchmark Factory. Automated Testing, Virtual Users, and Agents When you load test with Benchmark Factory, you are performing automated testing. Automated testing is the process of using software to test computer hardware or software implying that the software is used instead of actual users. The software acts as a "virtual" user. For example, say you would like to test your database with two hundred virtual users over a given period of time. Benchmark Factory lets you select two hundred virtual users and the length of time you wish to perform the test. Benchmark Factory uses Agents to deploy the virtual users. An Agent is a software routine that waits in the background and performs an action when a specified event occurs. One Agent can simulate thousands of virtual users (limited by hardware and workload characteristics) at a time. Each virtual user has their own connection to the system under test. Understanding Benchmarks A benchmark is a performance test of hardware and/or software on a system-under-test. Benchmark Factory provides the option of using industry standard benchmarks during the load testing process. Benchmarks measure system peak performance when performing typical operations. Benchmark Factory comes equipped with the following industry standard benchmarks:
l l

AS3AP Benchmark (page 223) Scalable Hardware Benchmark (page 224)

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

l l l l l

TPC-C Benchmark (page 226) TPC-C Benchmark (page 226) TPC-D Benchmark (page 229) TPC-E Benchmark (page 230) TPC-H Benchmark (page 231)

Understanding Benchmark Factory Terminology

The following provides a list of terminology required to understand the Benchmark Factory load testing process.
l

A script is the user's workspace that allows you to create and save jobs.
l

Scripts are created in the Script View. See "Script View" (page 38) for more information. When ready to run, jobs are submitted to the Jobs Queue View. See "Jobs Queue View" (page 39) for more information. The Load Scenario Wizard is the starting point for the creation of scripts. See "Load Scenario Wizard" (page 154) for more information.

A job contains load scenario, execution of a file, creation/deletion of benchmark objects, or a combination of each. A load scenario is an assembled test comprised of user scenarios and/or transactions. These tests can be run with multiple virtual users. A load scenario can be one of the following:
l

Mixed Workload: A Mixed Workload test runs for a specified time at each predetermined user load level. Each user will run a transaction mix based upon the weights defined on the transactions tab. For example, if a test has two transactions, A and B, with A having a weight of one and B having a weight of four, on average B will run four times for every time A is run once. The run order will be randomly generated for each user so they are not all running the same transaction simultaneously. That run order is used for that user each time the test is performed to ensure reproducible results. Goal Test: A goal test is used to find maximum throughput or response time goals. A transaction mix is executed at user load levels, determined by setting a beginning, ending, and interval value. When run, the specified goal criterion is evaluated at the end of each iteration and the test ends once the goal or maximum user load has been reached. Replay Test: A Replay Test runs multiple transactions with each one running independently on a specified number of users. The test will run until the defined number of executions for each transaction or a specified time limit is reached.

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

SQL Scalability Test: A SQL Scalability test executes each transaction individually for each userload and timing or execution period. For example, a test could have two transactions, A and B, and two userloads of 10 and 20, with an iteration length of one minute. Transaction A would execute continually for one minute at userload 10, then B would do the same. Next A will run at userload 20, followed again by test B, for a total time of 4 minutes.

Click here for load scenario creation procedures.

A user scenario is a series of one or more transactions to be executed in order to make a single transaction, normally associated with user behavior simulated against the systemunder-test. An Execute SQL Statement from a file User Scenario will execute SQL statements as defined in an XML file. See "User Scenario Wizard " (page 194) for more information. A transaction is a single unit of work in the testing process, used in load scenarios and user scenarios. For example, a SQL statement. Click here for procedures to create a transaction. Execute a file allows you to execute a file during the running of a job. See "Execute File Wizard" (page 153) for more information. Create or delete benchmark objects configures a system-under-test for industry standard benchmark tests to measure system performance. Each standard benchmark has been developed with specific system configuration requirements, that include tables, indexes, data, etc. See "Benchmark Objects Wizard" (page 150) for more information.

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

How Benchmark Factory Works

The following steps provide an overview of how Benchmark Factory components interact during the load testing process. 1. The BenchmarkFactory Console (page 32) implements the workload testing process and controls one or more distributed agent machines. Each agent machine can simulate thousands of users. Each simulated user executes transactions and records statistics. 2. Benchmark Factory Agent(s) (page 33)machines simulate virtual users. The Agents send transactions to the system-under-test. The Agents record statistics that includes how much data the resulting transaction contained and how long it took to get the results. At the end of an iteration, each agent machine reports its findings back to Benchmark Factory. 3. A server (system-under-test) is the database Benchmark Factory connects to. Benchmark Factory is server neutral and network protocol independent. Benchmark Factory uses vendor client libraries of the system-under-test. Any system that the client software can support, Benchmark Factory can support. Its only requirement is that Agent machines must be able to connect to a server through an appropriate manner and it supports the test system. 4. The Repository Manager (page 36) stores all testing data. 5. Results are viewed from Run Reports Node (page 207).

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

Create a Script With Benchmark Factory

The following provides a narrative discussion on how to create a script using Benchmark Factory. This process consists of following steps: 1. Set up Benchmark Factory 2. Create a Script 3. Create a Profile 4. Create a Job 5. Run the Script 6. Analyze Test Results Note: The Benchmark Factory Quick Starts provide procedures that introduce you to the Benchmark Factory environment. See "About the Quickstarts" (page 267) for more information. 1. Set up Benchmark Factory The Settings window displays default values used when creating new scripts. Changes to these settings affect only new load scenarios, not existing load scenarios. To Display Benchmark Factory Settings
l

Select Edit | Settings. The settings dialog displays.

The Settings window provides the following tabs:

l l l l l l

General Settings (page 125) Benchmarks Settings (page 127) Timing Settings (page 128) LatencySettings (page 131) Repository Settings (page 133) Agent Settings

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

2. Create a Script Using the Benchmark Factory graphical user interface, you can create scripts in the Script View. You open one script a time when using Benchmark Factory. See "BenchmarkFactory Console" (page 32) for more information.

All tasks required to complete a Benchmark Factory script can be accomplished using wizards. The first wizard you will use is the Load Scenario Wizard. The Load Scenario Wizard takes you through the steps required to create a script and complete a database load test. See "Load Scenario Wizard" (page 154) for more information.

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

The first step the Load Scenario wizard asks you to perform is to select a load scenario type from the load scenario dialog. Load Scenarios are an assembled load test comprised of:
l

User scenarios: A series of one or more transactions to be executed in order normally associated with a behavior to be tested on the system-under-test. These are the components used to build a load scenario. Transaction: A single unit of work in the testing process, such as retrieving a Web page, executing a SQL statement, writing a file, or sending an email.

After selecting the desired task type, the Load Scenario Wizard will guide you through the steps for script creation, depending on which type of task you select. 3. Create a Profile Profiles are sets of information that Benchmark Factory uses to communicate with the systemunder-test . This information includes, server name, IP address, and user name. You create profiles using the:

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

l l

Profile Creation Wizard: (See "Profile Creation Wizard" (page 175) for more information. Edit Profiles (See "Edit a Profile" (page 74) for more information.

4. Create a Job A job holds a load scenario, execution of a file, creation/deletion of benchmark objects, or a combination of each. Jobs are created in Benchmark Factory when a load scenario is added to a script. A script can hold as many jobs as you chose to create.

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

Change Script Properties Benchmark Factory script properties can be changed or adjusted to meet load testing requirement. You change properties by clicking on the node of the property you wish to change. For example, if you wish to change the user load test you click on the Load Scenario node. See About the Script Node for more information.

To change the User Load properties, select the User Load tab and select the desired number of users.

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

5. Run the Script After completing the wizards and setting in the desired properties, you run the job by selecting the Job Node and clicking the Submit Job icon , or right-clicking and selecting Submit Job. See "Submit Job Dialog " (page 184) for more information.

6. Analyze Test Results Benchmark Factory provides a number of metrics to analyze a system-under-test. These include:
l l l l l

Response, retrieval, and transactions times Transaction throughput in transactions per second (TPS) Amounts of data transferred in bites per second (BPS) Errors incurred Transaction Distribution

Benchmark Factory for Databases User Guide Understanding Benchmark Factory

Any user defined counters (Windows NT/2000 Performance Monitor) added using the Benchmark Factory Console.

Benchmark Factory provides the following testing results:

l l l

Real time statistics (See "Real-Time Statistics Node " (page 203) for more information.) Run Reports (See "Run Reports Node " (page 207) for more information.) Testing results exported to Excel (See "Export Test Results to Excel" (page 221) for more information.)

3
Benchmark Factory Components
BenchmarkFactory Console
The Benchmark Factory console implements the database workload testing process. This interface is where:
l

Scripts are created and initiated. A script is the user workspace that allows you to design and save jobs. Jobs are created and submitted to the Jobs Queue View. See "Jobs Queue View" (page 39) for more information. All script properties are set. See About the Script Node for more information. Settings can be made. See "About Settings" (page 125) for more information.About Scripts (page 234) Users can access test results. See "Benchmark Factory TestingResults" (page 198) for more information. Agents are managed. See "Agent(s)" (page 33) for more information.

l l

The Benchmark Factory Console contains the following views:

l l l l l

Script View (page 38) Jobs Queue View (page 39) Properties View (page 40) Agents View (page 40) Output View (page 40)

Benchmark Factory for Databases User Guide Benchmark Factory Components

Agent(s)
Benchmark Factory Agent(s) reside on each physical client machine and spawn multiple virtual-user sessions. Each Agent generates an information screen with tabs that contain a variety of Agent information. Benchmark Factory can control hundreds of Agent machines. Each Agent can simulate up to 1500 virtual users, depending upon hardware and test configuration. Virtual users simulate the load against the system-under-test. Each virtual user is a separate thread, acting independently of the other virtual users, with its own connection to the system-under-test. Each virtual user tracks its own statistics that includes transaction times and the number of times a transaction executes.

The Agent provides the follow tabs:

l l l l

Agent Test Status Tab Agent Virtual Users Tab Agent Transaction Tab Agent Output Tab

Benchmark Factory for Databases User Guide Benchmark Factory Components

Run Reports Node

Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page 212) for more information. Note: Three instances of Run Reports can be viewed at one time.

Cutting and Pasting of Run Reports

From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See Cut and Pasting Reports for more information.

Testing Server Throughput

Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:
l l l l l l l l

Transaction/Second (TPS) Transaction Time Bytes/Second (BPS) Rows/Second (RPS) Total Bytes Total Errors Total Rows Response Time

Benchmark Factory for Databases User Guide Benchmark Factory Components

Testing SQL Scalability

The Benchmark Factory transaction comparison graph provides users with a comparison of the individual transaction performance as they compare to each other. This is most useful when analyzing a SQL Scalability test.

Testing Real Application Clusters

Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of individual nodes.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Repository Manager
Note: If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark Factory will not work against this repository. The Repository is a database where all of the test results are stored. Benchmark Factory inserts test results into the repository and provides an easy way to access the data. By default, the Repository is a MySQL database that resides on the same machine as Benchmark Factory for Databases. The Repository can reside on another database server if required. Tto change the database select the Data Source Name of the ODBC connection for the new database. Note: If the database structure does not exist on the selected database, a prompt to create the structure will appear when OK is clicked. The supported repository database types are:
l l l l

Sybase Adaptive Server Anywhere Microsoft SQL Server MySQL Oracle

Benchmark Factory for Databases User Guide Benchmark Factory Components

Section Connection Parameters

Field/Icon Data Source Name ODBC Driver

Description Data Source name of the ODBC connection used to connect to the repository database. Current ODBC driver

User Name

The User Name used to log into the selected database.

Password

The Password associated with the user name used to log into the database.

Displays the ODBC connection information dialog for the selected data source.

Displays the ODBC Data Source Administrator dialog, used to add and edit ODBC connections.

Tests the connection of the currently selected ODBC Data Source.

Maintenance

Creates the repository objects on the selected database.

Deletes the repository objects on the selected database. Warning: This will delete all test results stored in the Repository. Launches the Data Migration Wizard. See "Data Repository Migration Wizard" (page 152) for more information.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Views
Script View
The Script View displays the currently open script and provides user workspace that allows you to design and save jobs. When ready to run, jobs are submitted to the Jobs Queue. The Script View Toolbar provides user functionality when creating or editing load testing scripts. Note: You must be in the script view to create jobs and edit jobs.

Script View Toolbar The Script View toolbar provides user functionality when creating or editing load testing scripts.

Creates a blank job. You must click on the Script Node to active the icon.

Displays the Load Scenario Wizard (page 154) Displays the User Scenario Wizard (page 194)

Benchmark Factory for Databases User Guide Benchmark Factory Components

Displays the Transaction Wizard (page 186) Displays the Benchmark Objects Wizard (page 150)

Displays the Execute File Wizard (page 153) Displays the Submit Job Dialog (page 184)

Moves an item up in the Script View

Moves an item down in the Script View

Jobs Queue View

The jobs view displays the currently open jobs. From the Jobs Queue, jobs are run, status can be viewed, and real-time testing results can be viewed. The Jobs toolbar provides user functionality when running a job.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Jobs Queue Toolbar Jobs Queue icons provides user functionality when running jobs. Deletes the selected job from the Jobs Queue. Saves the selected job to the Script View. See "Save the Selected Job to the Script View" (page 116) for more information.

Expands all jobs when the root is selected. Collapses all jobs when the root is selected.

Properties View
The Properties View allows you to monitor and managed scripts. Properties can be viewed for the:
l l

Script Node Properties (page 41) Jobs Queue Node Properties (page 55)

Agents View
The Agent view displays the status of connected agents.

Output View
The Output window displays errors and messages that occur outside of running a job. For example, a .dll error or re-registering a benchmark.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Properties
About the Properties View Benchmark Factory properties allow you to:
l l l

Adjust script parameters View script properties Access Benchmark Factory Run Reports See "Run Reports Node " (page 207) for more information. View realtime statistics (See "Real-Time Statistics Node " (page 203) for more information. Set and view latency See "About Latency " (page 118) for more information.

The following provides Benchmark Factory properties definitions:

l l l l

Script (See About the Script Node for more information.) Job (See "Jobs Queue Node Properties" (page 55) for more information.) Latency (See "About Latency " (page 118) for more information.) Benchmark Properties See "Benchmark Properties" (page 67) for more information.)

Script Node Properties Script Node The Script node displays the Benchmark Factory Home page and Help options.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Jobs Node Note: In the Jobs Queue view, not all properties are editable. The Jobs node provides the following tabs:
l l l l l

General Job Schedule Jobs Alerts Real-Time Counters Global Variables Node (page 56)

Load Scenario Node Note: In the Jobs Queue view, no properties are editable. The Load Scenario node provides the following tabs:
l l l

Transaction Tab Goal Test (page 172) User Load Tab (Measurement Intervals)

Benchmark Factory for Databases User Guide Benchmark Factory Components

l l

Timing Tab Options Tab-Mixed Workload, Goal, and Scalability Test (page 48)

User Scenario Node The User Scenario node properties allows you to:
l l l

Delete transactions Adjust transaction order Adjust latency (See "About Latency " (page 118) for more information.)

User Scenario From File The User Scenario From File node properties allows you to:
l l

Executing a SQL Statements from a File (page 44) Adjust latency (See "About Latency " (page 118) for more information.)

Benchmark Factory for Databases User Guide Benchmark Factory Components

Executing a SQL Statements from a File When large Oracle trace files are imported into Benchmark Factory, they are converted to editable XML files. To Edit SQL Statements from a file 1. Click Edit.

2. Click File | Open.

The Open dialog displays. 3. Navigate to and select the required file XML file. 4. Click Open.

Benchmark Factory for Databases User Guide Benchmark Factory Components

5. Under Session Details, click on the SQL statement you wish to edit. That statement displays in the SQL Statement view.

6. From the Prepare SQL tab, you can click in the upper right-hand corner to:
l

Run SQL Preview See "Running SQL Preview" (page 92) for more information. Launch the BFScript Wizard See "BFScript Wizard" (page 176) for more information.

Benchmark Factory for Databases User Guide Benchmark Factory Components

7. After you finish making the required edits, click File| Save to save your edits.

Transaction Node Note: In the Jobs Queue view, no properties are editable. The transaction node displays properties based on transaction type. To view the Transaction node Click the Transactions node in the Jobs Queue or Script View.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Options Tab-Replay Test The Repository Properties tab controls what information is saved in the Repository during the running of a test, and allows you to set error handling properties. The database where all Benchmark Factory information is saved. It contains information about the transactions stored in each benchmark DLL and is where all of the statistical information is stored about benchmark executions.

To change Repository properties 1. Click the Load Scenario node. 2. Click the Options tab.

Setting Repository Settings

Description

Save Real-Time Counter Information Save User Statistics

Saves Benchmark Factory real time counters and performance monitor counters to the repository during test. Saves user level statistics.

Save Distribution

Saves transactions per second and response time distribution.

Error Handling

Stops the test when an error is returned.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Options Tab-Mixed Workload, Goal, and Scalability Test The Options tab specifies Repository, error handling, file execution, and database checkpoint settings.

Section Repository Settings

Field/Icon Save Real-Time Counter Information Note: To change real-time counters please refer to Profile Properties Real-Time Counter Tab. Save User Statistics

Description Saves counters and performance monitor counters to the Repository during test. Sample rate is determined in job settings.

Saves user level statistics

Save Response Distribution

Saves transaction per

Benchmark Factory for Databases User Guide Benchmark Factory Components

Section

Field/Icon

Description second and response time per user distribution.

Log Transaction Results to disk

Logs the transaction results of a job to disk. Note: Checking this option imposes performance penalties.

Error Handling

Stop test after first error

When checked, the test is stopped when an error is reported. Name of the program to execute at the beginning of an iteration.

Execute the following program at the beginning of each iteration

File Name Note: This field accepts BFScripts.

Enforce Timeout

Enforces a timeout on the file being executed. Name of the program to execute at the end of an iteration. Enforces a timeout on the file being executed so that if the program does not complete in the specified time, it will be stopped, and the load scenario will continue. Initiates a database checkpoint at the beginning of a test iteration. Initiates a database checkpoint during a test iteration. Specifies the number of checkpoints to initiate.

Execute the following program at the end of each iteration

File Name Note: This field accepts BFScripts. Enforce Timeout

Database Checkpoints

Perform checkpoint at start of each test iteration

Perform checkpoints during each iteration

Number of checkpoints

Benchmark Factory for Databases User Guide Benchmark Factory Components

Run Reports Node Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page 212) for more information. Note: Three instances of Run Reports can be viewed at one time. Cutting and Pasting of Run Reports From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See Cut and Pasting Reports for more information. Testing Server Throughput Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:
l l l l l l l l

Transaction/Second (TPS) Transaction Time Bytes/Second (BPS) Rows/Second (RPS) Total Bytes Total Errors Total Rows Response Time

Benchmark Factory for Databases User Guide Benchmark Factory Components

Testing SQL Scalability The Benchmark Factory transaction comparison graph provides users with a comparison of the individual transaction performance as they compare to each other. This is most useful when analyzing a SQL Scalability test.

Testing Real Application Clusters Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of individual nodes.

Benchmark Factory for Databases User Guide Benchmark Factory Components

SQL Statement Node The SQL statement node displays individual SQL statements and allows you edit a SQL statement or use the BFScript Wizard. See "BFScript Wizard" (page 176) for more information.

Create/Delete Benchmark Objects Node Note: In the Jobs Queue view, no properties are editable. The Create/Delete Benchmark node allows you to either create the benchmark objects or delete the benchmark objects. Benchmark objects are used by standard benchmark load scenarios. Benchmarks can be added to a job using the Benchmark Objects Wizard. See "Benchmark Objects Wizard" (page 150) for more information.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Field Benchmark

Description Name of the benchmark whose benchmark properties are being edited.

Scale

The factor used to increase the amount of data added to the database. Valid benchmark scales are dependent upon the benchmark selected. For example, the AS3AP scales by a factor of 10 with valid scales of "1, 10, 20, 30..." and the TPC-C benchmark scales by a factory of 1, with valid scales of "1, 2, 3, 4..."

Displays the Benchmark Properties page and allows you to edit the benchmark scale and map database tables and indexes. See "Benchmark Properties" (page 67) for more information.

Always delete objects before create

Checking this box deletes benchmark objects before creating new ones. When unchecked, Benchmark Factory checks for the benchmark tables. If they exist, then the creation will be skipped.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Field

Description Note: The scale factor is not checked.

Execute File Node Note: In the Jobs Queue view, no properties are editable. The Execute File node allows you to place a file execution in a job as a sibling to a load scenario. The file executes during the running of a job.

Field Execute File

Description Type in or Select the File Name to execute: Name of the file you wish to execute. Note: This field accepts BFScripts. See "About Scripts" (page 234) for more information. Click to open a file.

Enforce Timeout

Places a time constraint on the number of seconds a file has to execute. The number of seconds a file has to execute.

Enter a number

Benchmark Factory for Databases User Guide Benchmark Factory Components

Field between 30 and 10000 [n] Secs.

Description

Jobs Queue Node Properties About the Jobs Node Some of the nodes in the Jobs Queue are the same as the Script view. Note:Not all nodes in the Jobs Queue are editable and provide property viewing only. The following nodes display:
l l l l l l l l l l

Jobs Queue Node Load Scenario Node (page 56) User Scenario Node (page 56) Transaction Node (page 57) Create/Delete Benchmark Objects Node (page 57) Run Reports Node (page 207) Run Status Node (page 203) Real-Time Statistics Node (page 203) Summary Node (page 205) Messages Node (page 64)

Jobs Node Note: In the Jobs Queue view, not all properties are editable. The Jobs node provides the following tabs:
l l l

General Job Schedule Jobs Alerts

Benchmark Factory for Databases User Guide Benchmark Factory Components

l l

Real-Time Counters Global Variables Node (page 56)

Global Variables Node From the global variable node, you can add or delete the required variables. To add a global variable: 1. Right-click to display the global variable dialog. 2. Enter the name and value. 3. Click Add.

Load Scenario Node Note: In the Jobs Queue view, no properties are editable. The Load Scenario node provides the following tabs:
l l l l l

Transaction Tab Goal Test (page 172) User Load Tab (Measurement Intervals) Timing Tab Options Tab-Mixed Workload, Goal, and Scalability Test (page 48)

User Scenario Node The User Scenario node properties allows you to:
l l l

Delete transactions Adjust transaction order Adjust latency (See "About Latency " (page 118) for more information.)

Benchmark Factory for Databases User Guide Benchmark Factory Components

Benchmarks can be added to a job using the Benchmark Objects Wizard. See "Benchmark Objects Wizard" (page 150) for more information.

Field Benchmark

Description Name of the benchmark whose benchmark properties are being edited.

Scale

Displays the Benchmark Properties page and allows you to edit the benchmark scale and map database tables and indexes. See "Benchmark Properties" (page 67) for more information.

Always delete objects before create

Checking this box deletes benchmark objects before creating new ones.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Field

Description When unchecked, Benchmark Factory checks for the benchmark tables. If they exist, then the creation will be skipped. Note: The scale factor is not checked.

Transaction/Second (TPS) Transaction Time Bytes/Second (BPS) Rows/Second (RPS) Total Bytes Total Errors Total Rows Response Time

Benchmark Factory for Databases User Guide Benchmark Factory Components

Run Status Node Run Status Node properties displays information on the job when it is running. You can monitor the status of the test, which userload its running, transactions per second (TPS), and a variety of other datapoints that give you insight into the actual performance your database provides under the userload being tested. Iteration Overruns Iteration overruns occur at the end of an iteration to allow time for all transactions submitted within the test iteration cycle to complete, so that all transaction statistics can be collected. For example, an agent may execute a transaction during the last five seconds of test iteration, if this transaction takes 15 seconds to complete, an iteration overrun of 10 seconds will occur. To view the Run Status node 1. Click the Run Status node in the Jobs Queue.

Real-Time Statistics Node The Real Time Statistics node provides you with real-time graphs and raw data. This data allows you to spot system-under-test issues that may be affecting server performance. The Real Time Statistics node displays real-time counter data. Right-clicking inside the graph displays a dropdown that allows you to change graph settings and view. To view the Real-Time Statistic node 1. Click the Real-Time Statistics node in the Jobs Queue.

Benchmark Factory for Databases User Guide Benchmark Factory Components

2. Select Click here to Add Datapoints. The Add Datapoints dialog displays.

3. Select the desired datapoints to view. 4. Click OK.

Benchmark Factory for Databases User Guide Benchmark Factory Components

5. To change graph views, right-click. See "Change Graph Views" (page 311) for more information.

Summary Node The Summary node displays user selected statistics for the job timing period following completion of an iteration. To view the Summary node 1. Click the Summary node in the Jobs Queue.

2. Click the datapoints Vs. Userload drop-down.

Benchmark Factory for Databases User Guide Benchmark Factory Components

3. Select the desired datapoints to view. 4. To change graph views, right-click. See "Change Graph Views" (page 311) for more information. 5. Select the desired option.

Messages Node The Messages node displays:

l l

Job status Any errors or warnings

Latency About Latency Latency is the speed in which transactions are sent to a server or the period between receiving results of one transaction and sending the next transaction to a serve. Benchmark Factory allows you to select the following latency models and distribution models: Latency Types No Delay (page 65) Distribution Models Absolute Distribution (page 66)

Interarrival Time (page 65)

Uniform Distribution (page 66)

Keying Time (page 65)

Negative Exponential Distribution (page 66)

Think Time (page 65)

Normal Distribution (page 67)

Poisson Distribution (page 67)

Benchmark Factory for Databases User Guide Benchmark Factory Components

No Delay No Delay means that transactions execute as fast as possible. As soon as one transaction is processed, the next transaction is issued against the server. In the case of a mixed workload test, each virtual user issues transactions as fast as possible. The No Delay option is used when the goal of the test is to stress the system to its limits, without concern for accurately simulating users. With No Delay specified, a relatively low number of users can stress the system to its limits. As such, there is no easy way to correlate N virtual users running with no delay to some number of real users.

Interarrival Time Interarrival Time derives its name from achieving a specific transaction rate as seen by a server (the rate at which transactions are arriving at a server.) When specifying Interarrival Time, Benchmark Factory is instructed, that regardless of how long a transaction actually takes to execute, to ensure that the transactions arrive at a server at a specific interval.

Keying Time Keying Time is used to simulate an amount of time performing data entry, before executing a transaction (entering information). In many cases, Keying Time is used with Think Time to provide a delay before and after a transaction execute. Selecting Keying Time inserts a fixed or variable delay before executing a transaction.

Think Time Think Time is used to simulate an amount of time "to think" about the results of a previous transaction. This could be time spent performing analysis on the results of a database query. Selecting Think Time inserts a fixed or variable delay after executing a transaction.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Latency Distribution Models About Benchmark Factory provides the following distribution models
l l l l l

Absolute Distribution (page 66) Uniform Distribution (page 66) Negative Exponential Distribution (page 66) Normal Distribution (page 67) Poisson Distribution (page 67)

Absolute Distribution Absolute is a fixed length delay. If a 2000 ms delay is specified for Keying Time (page 65) and a 3000 ms delay is specified forThink Time (page 65), each time the transaction should execute, Benchmark Factory waits 2000 ms, then fires the transaction, then waits an additional 3000 ms before deciding which transaction to execute next. If an Inter-Arrival model is chosen with a delay type of Absolute set to 2000 ms, Benchmark Factory marks the time, executes the transaction, and assuming the transaction finishes in less than two seconds, waits until two seconds from the marked time has elapsed before determining which transaction to execute next.

Uniform Distribution Selecting a Uniform delay instructs that random delay should be chosen, with an equal probability of being the minimum value, the maximum value, or any value in between. Uniform delays are chosen when it is suspected that the delay is highly random within a range or a minimal amount of statistical analysis has been performed to determine how the actual users react. Suppose a uniform Keying Time (page 65) is selected with a minimum value of 1000 ms and a maximum value of 1500 ms. If the transaction is executed more than 500 times, there is a high probability that each possible delay has been selected at least once. With the other delay types, this is not the case. If the same 2000 ms to 2500 ms uniform delay is set for Interarrival time, the tester essentially is setting the test so that a server sees the transaction every 2 to 2.5 seconds, instead of exactly two seconds as in the Absolute delay time.

Negative Exponential Distribution Similar to the Normal distribution, Negative Exponential inserts a random delay based on a mathematical model. A Negative Exponential distribution should be chosen when most users

Benchmark Factory for Databases User Guide Benchmark Factory Components

have latency of the mean latency or less. Negative Exponential is weighted heavily toward smaller latencies being selected relative to the mean and maximum latency.

Normal Distribution Normal distributions differ from Uniform delays in that most of the delays chosen by Benchmark Factory will be close to the average, but can vary by as much as 10% of the mean. While a Uniform delay is used when users have latencies within equal likelihood of being anywhere between two values, Normal distributions are chosen when all users fall within a range, but most of the modeled users have a latency close to the average latency.

Poisson Distribution A Poisson distribution is very similar to the Normal distribution and can be used most places where a Normal distribution delay could be used. The biggest difference between a Normal distribution and a Poisson distribution is that Poisson selects discreet values.

Benchmark Properties Benchmark Properties Benchmark scale factors are used to increase the size of the database testing tables during the creation of benchmark objects, and to correctly execute the benchmark transactions as needed to fully utilize the database objects created. The Benchmark Properties page displays information on:
l l l

Scales (page 68)Scales (page 68) Advanced (page 69) History Tables (page 69)

Benchmark Factory for Databases User Guide Benchmark Factory Components

Scales The scale dialog specifies the benchmark scale used to create table data.

Field Benchmark Scale

Description The factor used to increase the amount of data added to the database. Valid benchmark scales are dependent upon the benchmark selected. For example, the AS3AP benchmark scales by a factor of 10 with valid scales of "1, 10, 20, 30..." and the TPC-C benchmark scales by a factory of 1, with valid scales of "1, 2, 3, 4..."

Show Empty Tables Estimated Size

Displays any tables that will be created but not populated with data.

The approximate size that the tables will use in the database without indexes. Additional space is needed for indexes.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Advanced Advancer mapping provides views of the tables and indexes that allows you to determine where there tables and indexes will be stored in the database. This allows your database to run more efficiently. See "Advanced Creation Objects (Object Mapping)" (page 79) for more information.

History Tables History tables allows you to set the number of history tables to create. The TPC-B benchmark is made up of only one transaction that updates three tables and inserts a record into a history table. Inserting one record into one history table limits testing performance. The Benchmark Factory properties page allows the user to set the number of history tables to create during a test. The best ratio of history tables to virtual users is based on database configuration and hardware. The number of history tables to use is determined by the tester.

Benchmark Factory for Databases User Guide Benchmark Factory Components

Field Number of History tables

Description Allows you to set the number of history tables to create.

4
How Do I...
Overview
The How Do I section points you to the procedures required to perform load testing.

How do I...
l

Create a Load Scenario - This wizard walks you through the steps to create and run scripts and load scenarios. Create a User Scenario - This wizard walks you through the steps to create a user scenario. Create a Profile - This wizard walks you through the steps to hook up to the data base you are testing. Edit a Profile Submit a Job - This wizard walks you through the steps to submit a job to the Jobs Queue for testing. Use Run Reports Use Advanced Creation Objects Create Benchmark Objects for Load Testing This wizard walks you through the steps to create benchmark objects for load testing. Execute a File - This wizard walks you through the steps to execute a file outside of Benchmark Factory. Change Scale Factors Use Benchmark Factory Scripting Capabilities - This wizard walks you through the steps to use Benchmark Factory built-in functions for use in scripts and the load testing process. Use SQL Scalability Wizard Copy a Load Scenario to a Replay Test, Mix Test, or Goal Test Run SQL Preview Replace Child Latencies Perform a Goal Test Use Global Search and Replace Create a Customized ODBC Trace File Import Profiles from the Repository Run a Real Application Cluster (RAC) Workload

l l

l l l

l l

l l l l l l l l l

Benchmark Factory for Databases User Guide How Do I...

l l l l l

Save the Selected Job to the Script View Set Latency Set the Number of Users on an Agent Stagger User Startup Run Benchmark Factory with Multiple Agents

User Scenario Wizard

The User Scenario Wizard allows you to build user scenarios. A user scenario is a series of one or more transactions to be executed in order, normally associated with user behavior simulated against the system-under-test. From the User Scenario Wizard you can:
l l l l l

Import and Replay Load Scenarios From an Oracle Trace File Replay load from SQL Server trace table Import from a delimited text file Replay from an ODBC trace File Create a custom user scenario

Profile Creation Wizard

Overview
Profiles are used by Benchmark Factory to connect to the system-under-test. The profile contains information required to connect to the system-under-test, such as server name, IP address, and user name. The Profile Creation Wizard walks you through steps required to create a Benchmark Factory profile. To create a profile 1. Select Wizards | Profile Creation. 2. Click Next. 3. From the Driver drop-down list, select the required driver type. 4. Click Next. 5. Enter the User/Schema.

Benchmark Factory for Databases User Guide How Do I...

6. Enter the Password. 7. From the TNSDriver drop-down, select the required database. or Click the Direct tab and enter the required fields. 8. From the Connect as drop, select the required connection, Normal, SYSDBA, or SYSOPER. 9. Select the installed client if applicable. 10. Click the Verify connection before proceeding check box to verify the connection to the specified database.

11. Click Next. 12. Enter a profile name. 13. Click Finish.

Benchmark Factory for Databases User Guide How Do I...

Edit a Profile
To edit a profile 1. Click the Edit Profile icon . in the upper right-hand

2. Double-click on a profile to edit or click View Properties corner of the dialog.

3. The Edit Profiles dialog displays. The following options are provided.
l

Adds a profile Imports a profile Deletes a profile Displays profile properties allowing you to edit the profile

Submit Job Dialog

A job contains load scenario, execute file, creation/deletion of benchmark objects, or a combination of each. The Submit Job Wizard walks you through the process of adding a job to the jobs queue. See "Jobs Queue View" (page 39) for more information. To submit a job to the Jobs Queue 1. Select the Job node. 2. Right-click.

3. Select Submit Job. The Submit Job dialog displays.

Benchmark Factory for Databases User Guide How Do I...

The Submit Job dialog allows you to tailor your job needs and provides the following options: Section Profile Description Allows you to select profiles already created.

Displays the Profile Creation Wizard. See "Profile Creation Wizard" (page 175) for more information.

Displays the Profile setting dialog. See "General Tab" (page 144) for more information.

Actions

Allows you to: Run Job: Submits the job to the queue and runs the job. Run Job when queue becomes idle: Submits the job to the jobs queue and puts it in a ready state so that it will run when the currently ready or running job completes.

Benchmark Factory for Databases User Guide How Do I...

Section

Description

Run Job per schedule: Submits the next job scheduled. Start Agent and Run Job: Starts the agent and runs the job. Start Agent and Run Job when queue becomes idle: Starts the agent and runs the job when all previous jobs have finished. Start Agent and Run Job per schedule: Starts the agent and runs the next job scheduled. No action: Closes the dialog after you select Submit.

Comments

Allows you to add a comment to the job.

Alerts

Allows you to send an email, pager, or net-send operator alerts when the job completes. Allows you to add Job Counters to the job.

Job Counters

Stop job after first error

When checked, the job will stop if an error is encountered.

4. After filling in the desired parameters, click Submit.

Run Reports Node

Benchmark Factory for Databases User Guide How Do I...

Cutting and Pasting of Run Reports

From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See Cut and Pasting Reports for more information.

Testing Server Throughput

Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:
l l l l l l l l

Transaction/Second (TPS) Transaction Time Bytes/Second (BPS) Rows/Second (RPS) Total Bytes Total Errors Total Rows Response Time

Benchmark Factory for Databases User Guide How Do I...

Testing SQL Scalability

Testing Real Application Clusters

Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of individual nodes.

Benchmark Factory for Databases User Guide How Do I...

Advanced Creation Objects (Object Mapping)

The Advanced Creation Option allows you to set advanced properties for tablespaces and datafiles, used for the creation of standard benchmark tables and indexes. To create advanced objects 1. From the Database Type drop-down, select the desired database type. 2. Select the Tables or Indexes radio button. 3. Select the object(s) you wish to map.

4. Click Edit Options. The Mapping Options dialog displays. 5. Select the desired profile from the Load Tablespaces from Profile drop-down. 6. Select the desired options. 7. Click OK. 8. To save the mapping options, click Save. The Save Options File As dialog display. 9. Enter the desired name. 10. Click Save.

Benchmark Objects Wizard

The Benchmark Objects Wizard creates or clears the database objects (tables/indexes, etc.) needed to run industry standard benchmarks. The Benchmark Objects Wizard is able to run if you have a root node job, load scenario, Create/Delete Benchmark object, or Execute File node selected. The Create/Delete Benchmark object node will be added just below the item selected or as a child of the job.

Benchmark Factory for Databases User Guide How Do I...

The Benchmark Objects Wizard will:

l l l

Select the benchmark to create objects for Determine the scale of a benchmark Create an instruction to create the benchmark objects

To create a Benchmark Object Note: To create, delete, and modify database tables/indexes you must have the user login rights required. 1. Select the Job or Load Scenario node. 2. Select Wizards | Benchmark Objects. 3. Click Next. 4. From the drop-down, select the desired benchmark. 5. Click Next. 6. Select the desired benchmark scale in Benchmark Scale. When setting up a load test, Benchmark Factory allows you to change the scaling factor of the standard benchmarks provided. Scales factors increase the size of a database during the testing process allowing realistic testing. By the scale factors, the number of rows added to the tables increases, allowing for larger userloads to be used, placing a greater stress on the system under test. During load testing, you can run various test with numerous scaling factors to ensure your database performs at various performance levels. 7. The Advanced Creation Option allows you to set advanced properties for tablespaces and datafiles, used for the creation of standard benchmark tables and indexes. To add mapping options click Advanced Creation Option or proceed to step 19. 8. From the Database Type drop-down, select the desired database type. 9. Select the Tables or Indexes radio button. 10. Select the object(s) you wish to map.

11. Click Edit Options. The Mapping Options dialog displays. 12. Select the desired options.

Benchmark Factory for Databases User Guide How Do I...

13. Click OK. 14. To save the mapping options, click Save. The Save Options File As dialog display. 15. Enter the desired name. 16. Click Save. 17. Click OK. 18. Click Next. The Create/Delete benchmark objects dialog displays and provides the following options:
l

Create Benchmark Objects: Creates and initializes benchmark objects (tables, indexes, etc.) required to run the standard benchmark transactions. Always delete objects before create: Deletes existing benchmark objects (tables, indexes, etc.) before creating new ones. Note: Checking the "Always delete objects before create" check box will delete all benchmark objects in the selected database and then re-create the objects in the databases. When the checkbox is unchecked, Benchmark Factory checks for the benchmark tables. If they exist, then the creation will be skipped. The scale factor is not checked.

Delete Benchmark Objects: Deletes existing benchmark objects (tables, indexes, etc.).

19. Select the desired option. 20. Click Next. The Benchmark Objects Wizard completion dialog displays. 21. Click Finish. The Benchmark Object node creates and displays in the Script View.

Changing Scaling Factors

When setting up a load test, Benchmark Factory allows you to change the scaling factor of the standard benchmarks provided. Scales factors increase the size of a database during the testing process allowing realistic testing. By changing the scale factors, the number of rows added to the tables increases, allowing for larger userloads to be used, placing a greater stress on the systemunder-test. During load testing, you can run various test with numerous scaling factors to ensure your database performs at varying user load levels. Benchmark Factory provides the following scalable benchmarks:
l l l l l l

AS3AP TPC-B TPC-C TPC-D TPC-H Scalable Hardware

Benchmark Factory for Databases User Guide How Do I...

Then Benchmark properties page allows you to change scale factors properties.

To Change Scale Factors

You can set the load test scale factor when:
l l l l

Creating a new script Creating/delete a benchmark object, using the Benchmark Object Wizard In Benchmark Factory settings From the Benchmark Object's property page

Setting the Scale Factor in Benchmark Factory Settings Benchmark Factory settings allows you to set default scaling factors so you don't have to set it each time you create a new load test. To set a default scale factor 1. Select Edit | Settings. The settings dialog displays. 2. Click Benchmarks. 3. Double-click the desired benchmark. The Benchmark Properties dialog displays. 4. Set the desired scale in Benchmark Scale. 5. Click OK. 6. Click OK. Changing the Scale Factory from the Benchmark Object Property Page After creating the a benchmark object, you can edit that object to change the scale factor. To edit the scale factor for a Benchmark Object 1. Click on the desired Benchmark Object in the Script view.

Benchmark Factory for Databases User Guide How Do I...

2. In the properties view for the Benchmark Object, select Edit. The Benchmark Properties page displays. 3. Set the desired scale in Benchmark Scale. 4. Click OK.

Execute File Wizard

Overview
During the running of a workload, you may want to add an executable file (.bat or .exe) as part of a job. This gives you the flexibility of setting up your database before running a workload. For example, you could run a .bat file to set up a prepare SQL statement before running a workload. To execute a file 1. Select the Job or Load Scenario node. 2. Select Wizards | Execute File. 3. Click Next . The Execute File dialog displays. 4. Enter the path or navigate to the desired file. 5. Click the Enforce Timeout checkbox if desired. Enforce Timeout allows you to enter the desired number of seconds Benchmark Factory waits for the execution to complete before continuing the job. 6. Enter the desired number of seconds. 7. Click Next. The completion dialog displays. 8. Click Finish. The Execute File is created and displays in the Script View.

BFScript Wizard
Note: Controls with yellow background can be scripted. Benchmark Factory scripting allows you to randomize the load testing process. Scripting can be utilized by selecting the SQL Statement node and inserting the script in the properties view. A field has scripting capabilities if its right-click menu contains the Script Wizard option. The Script Wizard is a quick and easy way to use Benchmark Factory scripts. The Script Wizard consists of a script category and a script function. Each script function has a short description included and if applicable, its parameters.

Benchmark Factory for Databases User Guide How Do I...

There are two features within Benchmark Factory that assist you when using scripting capabilities:
l l

Script Wizard Script Assist

Click here for an overview of $BFScripts.

Script Wizard
1. Right-click the edit control. A drop-down display.

2. Select BFScript Wizard from the drop-down. The Script Wizard displays.

Benchmark Factory for Databases User Guide How Do I...

Script Assist
1. Type in the desired SQL statement. When $BF is typed the Script Assist is activated allowing you to insert the desired script.

Benchmark Factory for Databases User Guide How Do I...

2. Select the desired script. 3. Press Enter. The script displays. 4. Type in parameters as required.

SQL Scalability Wizard

Note: To run the SQL Scalability Wizard from Toad you must have Benchmark Factory loaded. The Benchmark Factory console does not have to be open to run the SQL Scalability Wizard.

Overview
The SQL Scalability test allows you to execute SQL statements, letting users spot potential issues not seen with a single execution. Users can run variations of a SQL statement generated by SQL Tuning in order to find the SQL that will perform the best under a load test. Creating a SQL Scalability load scenario requires the following steps. 1. Creating the SQL tuning connection 2. Entering the desired SQL statement 3. Running the statement using the in Benchmark Factory SQL Scalability Wizard Creating the SQL tuning connection To create the SQL Turning connection 1. In Quest Central or Toad, connect to an Oracle or SQL Server database. 2. From the Quest Central main toolbar select Tools|SQL Tuning to launch. Entering the desired SQL statement To enter the desired SQL statement 1. Enter the desired SQL statement in SQL Text. 2. Click the Optimize Statement icon. Clicking this icon executes multiple syntax transformation rules to produce a list of semantically equivalent SQL statements. Running the SQL statement using the in Benchmark Factory SQL Scalability Wizard 1. Click the Benchmark Factory drop-down icon and select the desired option. Three options are provided: 2. Test for Scalability-Tests the currently displayed SQL.

Benchmark Factory for Databases User Guide How Do I...

3. Test All for Scalability-Tests all SQL statements. 4. Test Selected for Scalability-Tests the selected SQL statements. 2. The Benchmark Factory SQL Scalability Wizard displays. Click Next. The Measurement Interval dialog displays. 3. Enter the desired user load. 4. Click Next. The Iteration Length dialog displays. 5. Enter the desired number of executions per iterations or executions per iteration. 6. Click Next. The Real World Latencies dialog displays. 7. Select the desired latency. 8. Click Next. The connection information dialog displays. 9. Enter the connection parameters. 10. Click Next. The Connection Agent dialog displays. Benchmark Factory Agent(s) reside on each physical client machine and spawn multiple virtual-user sessions. Each Agent generates an information screen with tabs that contain a variety of Agent information. Benchmark Factory can control hundreds of Agent machines. 11. If there are more than 20 users in your test, you will need to run an agent. Click Start Local Agent. 12. Click Next. The Benchmark Factory SQL Wizard completion dialog displays. 13. Click Finish. The SQL Scalability dialog displays. From this dialog, you can control, monitoring, and view SQL testing results. This dialog provides the following tabs and buttons:

OverviewTab
The Overview tabs provides transactions per second testing results for individual user loads and iterations.

Benchmark Factory for Databases User Guide How Do I...

Transactions
The Transactions view provides the following statistics on the SQL statements being run:
l l l l l l l

Executions Rows Bytes Errors Average Transaction Time Minimum Transaction Time Maximum Transaction Time

2. Real-Time The Real Time Statistics provides real-time graphs and raw data. This data allows you to spot system-under test issues that may be affecting server performance. Right-clicking inside the graph displays a drop-down that allows you to change graph settings and view.

Benchmark Factory for Databases User Guide How Do I...

Summary
The Summary graph provides real-time and raw data summary graphs. The Summary graph displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows you to change graph settings and view.

Messages
The Output window displays messages about the job being run.

Benchmark Factory for Databases User Guide How Do I...

2. Run-Runs the test. 3. Stop-Stops a test. 4. Benchmark Factory-Launches the Benchmark Factory for Databases console. 5. Run Reports-Launches Benchmark Factory Reports.

Copy a Load Scenario to a Replay, Mix, Goal, or Scalability Test

Load scenarios can be quickly converted from one type test to another by using the "copy" command. A load scenario is an assembled test comprised of user scenarios and/or transactions. These tests can be run with multiple virtual users. A load scenario can be one of four types: mix test, goal test, replay test, or scalability. In Benchmark Factory you can copy a:
l

Replay test A Replay Test runs multiple transactions with each one running independently on a specified number of users. The test will run until the defined number of executions for each transaction or a specified time limit is reached.

Mixed test A Mixed Workload test runs for a specified time at each predetermined user loads level. Each user will run a transaction mix based upon the weights defined on the transactions tab. For example, if a test has two transactions, A and B, with A having a weight of one and B having a weight of four, on average B will run four times for every time A is run once. The run order will be randomly generated for each user so they are not all running the same transaction simultaneously. That run order is used for that user each time the test is performed to ensure reproducible results.

Benchmark Factory for Databases User Guide How Do I...

Goal test A goal test is used to find maximum throughput or response time goals. A transaction mix is executed at user load levels, determined by setting a beginning, ending, and interval value. When run, the specified goal criterion is evaluated at the end of each iteration and the test ends once the goal or maximum user load has been reached.

Scalability A SQL Scalability test executes each transaction individually for each userload and timing period. For example, a test has two transactions, A and B, and two userloads of 10 and 20, with an iteration length of one minute. Transaction A would execute continually for one minute at userload 10, then B would do the same. Next A will run at userload 20, followed again by test B, for a total time of 4 minutes.

Copying a load scenario creates a new load scenario containing all compatible settings, user scenarios, and transactions from the original test. To copy a Load Scenario to a Replay Test, Mix Test, or Goal test 1. Right-click the desired Load Scenario in the Script View. A drop-down displays. 2. Select the desired type of test you want to copy to. 3. The test creates and displays in the Script View.

Running SQL Preview

SQL Preview allows you to preview the results of individual SQL statements while editing a SQL transaction. To run SQL Preview 1. Double-click on a user transaction in the Script View. 2. Click the SQL Query icon . The SQL statement displays.

3. Select the appropriate profile from the Profile drop-down list.

4. Type the appropriate maximum number of rows to preview into Maximum Returned, or check the All box to preview all rows. 5. Click OK. The SQL Preview window displays.

Benchmark Factory for Databases User Guide How Do I...

6. Review the information. 7. Click Close.

Replace Child Latencies

Latencies are used to control the rate that transaction execute. Replace child latencies is used to quickly change all of the latencies of the direct children on a node. For example, when an Oracle trace file is imported, the latencies are automatically set to replay the transactions at their original rate, but changing these to a set value can replay the transactions in a shorter period of time. To replace Child Latencies 1. Right-lick on the desired load scenario. A drop-down displays. 2. Select Replace Child Latencies. The Replace Child Latencies Dialog displays.

3. Select the desired latencies.

Benchmark Factory for Databases User Guide How Do I...

4. Click OK. The latencies replace the previously set latencies for the children of the selected Load Scenario or User Scenario. Note: The latencies grandchildren of the selected node will not be changed.

Goal Testing Quick Start

Goal tests are used to determine the maximum transaction per second (TPS) or userload of a system. When performing a goal test, Benchmark Factory provides the following options: Find maximum TPS: After each iteration completes, the TPS value is compared with the previous two testing iterations. If the TPS value has dropped for the last two values, then the maximum TPS has been discovered and the test stops. The maximum TPS is dictated by the users system. Benchmark Factory submits the workload to the server and measures the TPS that the server is experiencing for that workload. The users system dictates how fast the workload will be process by the database server.

Benchmark Factory for Databases User Guide How Do I...

Find maximum userload where response time is less than [n] ms: This option allows you to execute the transaction mix at each userload until the response time for an iteration is less than the entered value.

Purpose of this Quick Start

This quick start presents a load testing methodology to run a goal test that finds the maximum Transactions Per Second (TPS) of a database, by adding a SQL Statement Transactions to the goal test, then viewing real-time and stored testing results to determine if your database lives up to the required performance parameters. A Goal Test uses a transaction mix A transaction is a single unit of work in the testing process, used in load scenariosand user scenarios. For example, a SQL statement. based on userload intervals. The userloads are determined by setting a beginning, ending value, and interval value. This value specifies an interval to increase the number of users tested for each iteration of the Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration, and the test ends once the goal or maxim user load has been reached.

Actions Performed During This Quick Start

l l l l l l l l

Creating a blank goal testing script Set script timing Add a transaction to the goal test Create Benchmark Objects Submit the job to run View Realtime Results View Run Reports Results Saving the Script

Creating a Blank Goal Testing Script 1. Start Benchmark Factory if required. To start Benchmark Factory a. Click the Start button from the Windows task bar. b. Select Programs Quest Software Benchmark Factory for Databases Benchmark Factory for Databases. Benchmark Factory launches. The Run Quick Start Dialog displays. Launch the Quick Start if desired.

2. Select the Script tab in the Script View.

Benchmark Factory for Databases User Guide How Do I...

3. Select Wizards | Load Scenario. The Load Scenario Wizard displays. The Load Scenario Wizard is the starting point for all Benchmark Factory scripts. The Load Scenario Wizard walks you through the steps to build a complete load testing script. 4. Click Next. The Select Load Scenario dialog displays. 5. Click the Create a custom load scenario (Scalability, Goal...). 6. Click Next. The Test Type dialog displays. 7. Click the Goal radio button. 8. Click Next. The Goal Page displays. The Goal page allows you to:
l

Find maximum TPS: Selecting this option will show how the maximum TPS can be reached during a load test. Find maximum userload where response time is less than [n]ms: Selecting this option shows how a maximum userload can be reached when response time is less than a keyed in value.

Benchmark Factory for Databases User Guide How Do I...

2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 14. Click Create a new Benchmark Factory Script file. 15. Click Next. The wizard completion dialog displays. 16. Click Finish. The Scripts creates and displays in the Script View. Setting Script Timing 1. Click the Max TPS (1-5 by 1) node.

2. Click the Timing tab. 3. Enter the desired test length in Total Test Length, for example, 1 minute.
4. Move the slider bar to select the Pre-Sampling and Sampling times . ThePre-sampling time is the length of time virtual users execute transactions A single unit of work in the testing process. For example, a Web page retrieval, a SQL statement, or an email message. These are the components used to build a load scenario. in order to reach a steady state before gathering statistics. Sampling time is the length of time used to collect statistics while virtual users are executing transactions.

Benchmark Factory for Databases User Guide How Do I...

5. Click the Start all users as quickly as possible radio button. Adding a Transaction to the Goal Test 1. Click the Max TPS (1-5 by 1) node.

2. Select Wizards | Transaction. The Transaction Wizard displays.

3. Click Next. The Define Transaction dialog displays. 4. Select Create SQL Statement Transaction. 5. Click Next. The SQL Statement dialog displays. 6. Enter the desired SQL statement. In this quick start, select * from a_updates where akey = $BFRand(30) is entered. As you type a SQL statement, the $BFScript Wizard displays and auto enters a BFScript.

Benchmark Factory for Databases User Guide How Do I...

7. Click Next. The Real World Latencies dialog displays. 8. Select Keying Time. 9. From the Keying Time drop-down select Normal. 10. In Mean enter 3000 milliseconds.

11. Click Next. The wizard completion dialog displays. 12. Click Finish. The transaction creates and displays in the Script View.

Benchmark Factory for Databases User Guide How Do I...

100

Creating Benchmark Objects 1. In order to create the a_updates table used in the SQL statement, Benchmark Objects must be created. Click the Max TPS (1-5 by 1) job node. 2. Right-click. A drop-down displays. 3. Select Add Create/Delete Benchmark Objects. The Benchmark Objects Wizard displays.

4. Click Next. 5. From the drop-down, select the AS3AP standard benchmark. 6. Click Next. The Benchmark Scale displays. 7. Set the desired scale in Benchmark Scale. 8. Click Next. The Create/Delete benchmark objects dialog displays. 9. Click Next. The wizard completion dialog displays. 10. Click Finish. 11. You must create the benchmark objects before running the script. Right-click Create Objects for AS3AP. 12. Select Move Up.

Benchmark Factory for Databases User Guide How Do I...

101

13. Move the Benchmark Object to the top node in the script view.

Submitting the job to run 1. Click the Max TPS(1-5 by 1) job node.

Benchmark Factory for Databases User Guide How Do I...

102

2. Select Wizards | Submit Job Wizard. The Submit Job dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Displays the Profile setting dialog.

Actions

Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Comments

Allows you to add a comment to the job.

Benchmark Factory for Databases User Guide How Do I...

103

Section

Description

Alerts

Allows you to send an email, pager, or net-send operator alerts.

Job Counters

Displays the Job Counter dialog.

Stop job after first error

Stops the job if an error is encountered.

3. Fill in the required parameters. 4. Click Submit. The job runs and displays in the Jobs Queue. Viewing Realtime Results 1. Click the Message node. The Message node displays Job status, error messages, and general information.

2. Expand the Run Status node. Run Status provides a real-time status on the test currently running in the Job's Queue. 3. Click the Realtime Statistics node. The Real Time Statistics provides you with real-time graphs and raw data. This data allows you to spot system-under test issues that may be affecting server performance. The Real Time Statics node displays real-time counter data. 4. Click the Summary node. The Summary Graph displays. The Summary page displays user selected statistics for the job timing period following completion of an iteration. 5. Click the Transaction Graph node. The Summary Graph displays. The transaction node displays properties based on transaction type.

Benchmark Factory for Databases User Guide How Do I...

104

Viewing Run Reports Results 1. When the job completes, click the Run Reports node.

2. Select the desired run. 3. Double-click. The Run Reports opens to the Run ID node. This view contains an summary of load scenarios statistics. 4. Expand the Summary Graphs node. Summary graphs provide a wide array of graphs that allow you to view in detail your testing results. 5. Expand the Results node. Results provide information on individual user load and agent load testing activity. 6. Close Run Reports when finished reviewing the load scenario. Saving the Script 1. Click the Script tab. 2. Select File |Save. The Save As dialog displays. 3. Enter the desired script name in File Name. 4. Click Save.

Use Global Search and Replace

Global Search and Replace allows you to search for and replace text from the selected item and all of its children. To use Global search and replace 1. In the Script View, right-click the User Scenario, Load Scenario, or Transaction node you wish to search and replace. 2. Select Global/Search and Replace. The Global Search and Replace dialog displays. 3. Enter the desired parameter in Find What. 4. Enter the desired parameter in Replace With.

Benchmark Factory for Databases User Guide How Do I...

105

5. Check Match Case or Match Whole word if desired. 6. Click Replace.

Creating an ODBC Trace File

To create an ODBC trace file 1. Click the Start button from the Windows taskbar. 2. Select Settings | Control Panel. The Control Panel window displays. 3. Double-click on Administrative Tools. The Administrative Tools window displays. 4. Launch Data Source (ODBC). The ODBC Data Source Administrator window displays. 5. Click the Tracing tab. 6. Click the Start Tracing Now button. The Start Tracing Now button changes to Stop Tracing Now button. 7. Run the ODBC transactions to be traced. 8. When the transactions complete, click on the Stop Tracing Now button. 9. Click OK.

Import Profiles from the Repository

When a profile is used to run a job, a copy of the profile is placed in the Run Reports Repository. When needed, this copy can be imported into the current Benchmark Factory Profile list. To import a Profile from the Repository 1. Click Edit Profiles. The Edit Profiles dialog displays. 2. Click the Import Profile button 3. Check the profile to be imported. 4. Click OK. . The Import Profile from Repository displays.

Benchmark Factory for Databases User Guide How Do I...

106

Real Application Clustering (RAC) Quick Start

Overview
An Oracle Real Application Cluster (RAC) combines several independent servers/nodes into one database system. RAC is a clustered database solution that provides two major functions: scalability and availability to a business continuum. Scalability is a relative term, based on a simple rule that as the number of users accessing the system increases, the RAC configuration should be able to handle increased activity. Availability is the ability of the system to provide continuous service when one or more of the components in the cluster fail. Two important criteria used to achieve these goals focus on the cluster being able to load balance across all nodes, and when one or more nodes in the cluster fails, the users failover. When a node in an Oracle RAC cluster fails, an Oracle database and the remaining nodes continue to run, providing uninterrupted service to users. Oracle RAC distributes database demands across several nodes, performing the function of load balancing. When more capacity is required, additional nodes can be added with minimal effort. Organizations spending time and money setting up a RAC cluster must ensure their systems have the stability to withstand user environment demands when adding nodes. An Oracle Real Application Cluster (RAC) combines several independent servers/nodes into one database system. RAC is a clustered database solution that provides two major functions: scalability and availability to a business continuum. Scalability is a relative term, based on a simple rule that as the number of users accessing the system increases, the RAC configuration should be able to handle increased activity. Availability is the ability of the system to provide continuous service when one or more of the components in the cluster fail. Two important criteria used to achieve these goals focus on the cluster being able to load balance across all nodes, and when one or more nodes in the cluster fails, the users failover. When a node in an Oracle RAC cluster fails, an Oracle database and the remaining nodes continue to run, providing uninterrupted service to users. Oracle RAC distributes database demands across several nodes, performing the function of load balancing. When more capacity is required, additional nodes can be added with minimal effort. Organizations spending time and money setting up a RAC cluster must ensure their systems have the stability to withstand user environment demands when adding nodes.

Purpose of this Quick Start

This Quick Start presents a load testing methodology and steps required to implement, manage, and perform full diagnostic capabilities of the Oracle RAC cluster, in order to obtain optimum user service times regardless of the demands placed on the RAC environment.

Actions Performed During This Quick Start

l l l l

Setting up the tnsnames.ora file Creating the Benchmark Factory cluster profile Running the RAC Load Test Importing an Oracle Trace File

Benchmark Factory for Databases User Guide How Do I...

107

Viewing RAC Testing Results

Setting up the tnsnames.ora file Navigate to the tnsnames.ora file located in Oracle-Home\network\admin. Transparent Network Substrate (TNS) is Oracle's networking architecture. TNS provides a standard application interface that enables network applications to transparently access underlying network protocols. Open the file in a text editor. Edit the tnsnames.ora file to set up clustering connections. For the first entry, the Net Service Name, in this example MyTNS, is used to define the root Net Service Name. The root Net Service Name is not used when running a test. The remaining Net Service Names must be named with the root and a number. In this example, MyTNS is the root and MyTNS1, MyTNS2, and MyTNS3 will be used when the test is run. The "MyTNS" and "MyTNS1" connections are duplicates and are required for Benchmark Factory to perform load testing. MyTNS2 and MyTNS3 are for the other two machines in the cluster. Figure 1 provides an excerpt from a tnsnames.ora file set up for a three node cluster.

Benchmark Factory for Databases User Guide How Do I...

108

Figure 1-tnsnames.ora file set up for a three node cluster. Creating the Benchmark Factory Clustering Profile The profile contains information required to connect to the system-under-test, such as server name, IP address, and user name. To create the Benchmark Factory Clustering profile
1. Start Benchmark Factory if required . To start Benchmark Factory a. Click the Start button from the Windows task bar. b. Select Programs Quest Software Benchmark Factory for Database Benchmark Factory for Databases 5.5. Benchmark Factory launches. The Run Quick Start Dialog displays. Launch the Quick Start if desired. 2. Select the Script tab in the Script View. 3. Click Wizards | Load Scenario. 4. Click Next . The Load Scenario dialog displays. 5. Click Validate Oracle RAC Cluster Scalability. 6. Click Next . The Profile Selection dialog displays. 7. Click New. The Profile Creation Wizard displays. 8. Click Next . The driver selection dialog displays. 9. From the driver drop-down, select Oracle. 10. Click Next . The Connection Information dialog displays. 11. Fill in the required parameters.

12. Click Next . The Profile Name dialog displays. 13. Enter the desired name. 14. Click Next . The wizard completion dialog displays. 15. Click Finish. The Profile Selection dialog displays. 16. Click Next . The Edit Scale dialog displays.

Benchmark Factory for Databases User Guide How Do I...

109

17. Click the TPC-C Benchmark Edit button. The Benchmark 'TPC-C' Properties page displays. 18. Select the desired Benchmark Scale. The larger the scale selection, the larger the database that will when created. It is suggested that a scale of 1 be selected, unless you desire a larger database to test with.

Note: Clicking the Advanced tab allows you to map benchmark objects. For these procedures, please see Advanced Creation Objects (Object Mapping). 19. Click OK.
20. Click the TPC-H Benchmark Edit button. 21. Select the desired benchmark scale. The TPC-H benchmark creates large testing databases. A scale of 1 is suggested. 22. Click OK. 23. Click Next . The Node Selection dialog displays. 24. Enter the desired number of nodes in Nodes. In this example, 10 nodes are selected. 25. Under Method, click the Benchmark Factory radio button. 26. Click Next . The Measurement Intervals dialog displays. 27. Enter the desired User Load. 28. Click Next . The Enter Script Name dialog displays and provides the following options:
l

Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with.

29. Select Create a new Benchmark Factory script. 30. Enter the desired name. 31. Click Next. The wizard completion dialog displays.
32. Click Finish. The script creates and displays in the Script View.

Benchmark Factory for Databases User Guide How Do I...

110

Running the RAC Load Test Benchmark Factory allows you to replay production workload and SQL scripts in different database environments, and places enormous stress on a database. By replaying production activity in a test database, potential performance problems can be identified before occurring in production. A system typically breaks under extreme load. By identifying system capacity and performance bottlenecks before they occur, Benchmark Factory facilitates proactive testing, which in turn reduces downtime, development costs, and potential loss of revenue. Benchmark Factory allows you to import a variety of user loads that include:
l

Industry standard benchmark to measure the performance of a system-under- test. Benchmark Factory provides the AS3AP, Scalable Hardware, TPC-B, TPC-C, TPC-D, and TPC-H standard benchmarks. Oracle trace files with bind variables. Imports from various tools such as Quest Performance Analysis SQL from the repository for replay.

l l

Importing an Oracle Trace File After creating the RAC script, you can enhance your script by importing an Oracle trace file workload by selecting the trace file, applying filtering options, and then replaying the file. When Benchmark Factory imports the Oracle trace activity it maintains concurrency between transactions and inter-transaction timings that simulate a production environment. Activity as it happened in production can be simulated in a test database using an unlimited number of concurrent users. Note: To create an Oracle trace file which includes bind parameters, a level number of 4 must be used. For example, to start an Oracle trace for a user session, use the following command: ALTER SESSION SET EVENTS '10046 trace name context forever, level 4' Up to this point, you have created the clustering profile and a script that contains the TPC-C and TPC-H benchmarks. Next you import trace files required to run the job. 1. In the script created, select the Script node. This allows the Oracle trace file import load scenario to be placed in its own job.

Benchmark Factory for Databases User Guide How Do I...

111

2. Select Wizards|Load Scenario. 3. Click Next. 4. Click Replay load from an Oracle Trace file. 5. Click Next. The Oracle 8i/9i/10g/11g Trace Input dialog displays. 6. Click on the Add Trace button. The Select File Location window displays. 7. Navigate to and select the appropriate trace file(s). 8. Click Open. The trace file displays under the Trace File Name column. 9. Repeat steps 6 8 until all trace files are added. 10. Click Next. The Oracle Trace Activity window displays. 11. Click the appropriate Date Range radio button. 12. If appropriate, select the "From" and "To" dates and times. 13. Uncheck the Include system activity (SYS USER) box to include system activity. Note: It is recommended to not check the Include system activity (SYS USER) check box. This imports Oracle SQL statements routinely run by Oracle for maintenance reasons and is not required for the load testing process. 14. Click Next. The Enter Script Name dialog displays and provides the following options: 15. Create a new Benchmark Factory Script file-Selecting this option allows you to create a and save a completely new script. 16. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 15. Check the Add to existing Benchmark Factory Script file radio button. 16. Click Next. The wizard completion dialog displays. 17. Click Finish. The trace files create and display in the Script View.

Benchmark Factory for Databases User Guide How Do I...

112

Running the Job Since the Benchmark Factory node balancing method was selected in the RAC wizard, each job has been assigned a profile specifically designed to test specific nodes by adding them to the testing process. Benchmark Factory sets up a load test with a single instance and individual jobs for groups of nodes as they are added to the testing process. This allows you to first run the single instance node and gather results. Then you can tune that nodes and make the same changes to your other nodes. Next you can add your nodes to the testing process in small groups, allowing you to easily see if issues occur due to the additions of a group of nodes. 1. Select the Single Instance node.

Benchmark Factory for Databases User Guide How Do I...

113

2. Right-click. A drop-down displays. 3. Select Submit Job. The Submit Job Dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Displays the Profile setting dialog.

Actions

Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Benchmark Factory for Databases User Guide How Do I...

114

Section Comments

Description Allows you to add a comment to the job.

Alerts

Allows you to send an email, pager, or net-send operator alerts. Displays the Job Counter dialog.

Job Counters

Stop job after first error 4. Fill in the required parameters.

Stops the job if an error is encountered.

5. Click Submit. The job runs and displays in the Jobs Queue. 6. To run the jobs designed to test the remaining nodes created for this test, repeat steps 1-6. Viewing RAC Testing Results Benchmark Factory provides a wide array of graphs to interpret your RAC testing results. Individual nodes testing results can be viewed for comparison. 1. Select Tools Run Reports. Run Reports displays.

Benchmark Factory for Databases User Guide How Do I...

115

2. You can select a single test or multiple test to view testing results. In this example, multiple tests are selected to view the RAC testing results.

3. Click Show Test Results. Run Reports Displays.

4. To view RAC testing results for the nodes selected, click Cluster Performance Graph.

Benchmark Factory for Databases User Guide How Do I...

116

5. To view other testing results, click the desired result from the User Load drop-down.

Save the Selected Job to the Script View

Benchmark Factory allows you to save jobs in the job queue back to the Script View from the Jobs Queue. This provides an easy way to make changes to existing load tests. To save the selected job to the Script View 1. Select the desired job in the Jobs Queue. 2. Right-click. A drop-down displays. 3. Select Save to Script. 4. The job is placed in the Script View.

Schedule A Job
Benchmark Factory allows you to run a job immediately after creation or schedule a job to run at a future time. Scheduling allows you to set predetermined dates, recurrences, daily frequencies, and durations of jobs. Note: When using the recurring option, use Ready Job in place of Run Job.

Benchmark Factory for Databases User Guide How Do I...

117

Click here for definitions on scheduling parameters. To schedule a job 1. Click the Job node. 2. Click the Schedule tab. 3. Check the Enabling check box. 4. Enter the desired date you want to start the job in On date. 5. Enter the desired time in At time. 6. If you wish only to run the job once, submit the job. If you wish to run the job on a recurring basis, proceed to step 7.

7. Check the Recurring check box. 8. Click the desired radio button in Occurs. 9. Set the daily frequency in Every. 10. Enter the desired daily occurrence. 11. Click the desired Daily Frequency radio button. 12. Enter the desired parameters. 13. Click the desired duration radio button. 14. Enter the desired parameters.

Benchmark Factory for Databases User Guide How Do I...

118

15. Submit the Job if it is in the Script View or Ready the job if it is in the Jobs Queue by clicking the Ready Job button on the Jobs Toolbar.

About Latency
Latency is the speed in which transactions are sent to a server or the period between receiving results of one transaction and sending the next transaction to a serve. Benchmark Factory allows you to select the following latency models and distribution models: Latency Types No Delay (page 65) Distribution Models Absolute Distribution (page 66)

Interarrival Time (page 65)

Uniform Distribution (page 66)

Keying Time (page 65)

Negative Exponential Distribution (page 66)

Think Time (page 65)

Normal Distribution (page 67)

Poisson Distribution (page 67)

Benchmark Factory for Databases User Guide How Do I...

119

Set the Number of Users on an Agent

Using the maximum number of agents on an Agent provides better utilization of machine configuration. For example, you can run one machine with ten users and another with 100 users. This allows light weight machines to be used with more powerful machines. You can set the maximum number of users on an Agent using one of the following procedures:
l l

From Agent Settings From the Agent View in Benchmark Factory

To set the number of users from Agent Settings 1. On the Agent menu select Options|Settings. The Agent Settings dialog displays. 2. Enter the desired number of users in Max. Virtual Users.

3. Click OK.

Benchmark Factory for Databases User Guide How Do I...

120

To set the number of users from the Agent View in the Benchmark Factory Console 1. Right-click on the Agent in the Agent view of Benchmark Factory. A drop-down displays. 2. Select Settings. The Agent Settings dialog displays.

3. Enter the desired number of users in Max. Virtual Users. 4. Click OK.

Stagger User Startup

When running a load test, you have the option of staggering user startup. This allows you to adjust startup loads if, for example, you are having issues with overloading of servers, when all of the users attempt to start at the same time. To stagger user startup 1. Click the Load Scenario node. 2. Click the Timing tab. 3. In the User Startup parameters select the desired parameters.

l l

Start all users as quickly as possible: Starts all users immediately after a test begins. Start all users at even intervals for: Sets the length of time to start the users in. The value divided by the number of new users of new users to start in the iteration, determines how often a user is started. Start a new user every: Starts a new user, then waits the entered number of seconds before starting the next user.

Benchmark Factory for Databases User Guide How Do I...

121

Running Benchmark Factory with Multiple Agents

To configure Benchmark Factory for Databases with multiple Agents you must:
l

Load the proper client libraries (i.e. Oracle, SQL server, etc) on the Benchmark Factory Console and Agent Machine. Install Benchmark Factory for Databases on the console and agent machines.

Benchmark Factory Console

Benchmark Factory allows you to run multiple Agents for load testing. There is no required setup for the Agents on the Benchmark Factory Console. You view the Agents results as they run from the Benchmark Factory Console. To view the Agents from the Benchmark Factory Console: 1. Click Start | All Programs | Benchmark Factory for Databases | Benchmark Factory for Databases. 2. Select View | Agent. The Agent view displays. Statistics from All Agents connected to the Benchmark Factory Console will display here.

Benchmark Factory Agent

Each Benchmark Factory Agent must be configured with the address of the Benchmark Factory Console. Each Agent sends load testing results back to the Benchmark Factory Console. To configure Benchmark Factory Agents 1. Click Start | All Programs | Benchmark Factory for Databases | Agent. 2. Select Options | Settings. 3. In Machine Name/IP address, enter the address of the Benchmark Factory Console Machine. For example, com123456. 4. Click OK. 5. Repeat this procedure for each Agent machine

Running a Job With Multiple Agents

To run a job with multiple agents 1. Create the desired job. 2. Start the agents and connect them to the console. 3. Run the job.

Benchmark Factory for Databases User Guide How Do I...

122

4. As the job runs, all connected Agents will display in the Agent View of the Benchmark Factory Console.

5
Creating Load Scenarios
Load Scenario Wizard
Overview
Note: To create load scenarios, you must be in the Script View. The Load Scenario Wizard is a starting point for creating Benchmark Factory scripts and walks you through the steps to build a complete workload testing script. Note: Only one Load Scenario can be run at a time. From this wizard you can test:
l

Industry standard benchmark (TPC-C, TPC-E, TPC-H...) (See "Industry Standard Benchmark (TPC-C, TPC-H...)" (page 155) for more information.) Caputre and Replay Oracle Workload (See Capture and Replay Oracle Workload for more information.) Validate Oracle RAC Cluster Scalability (See "Validate Oracle RAC Cluster Scalability" (page 156) for more information.) Test Database Scalability (See "Test Database Scalability" (page 157) for more information.) Test SQL for Scalability (See "Test Database Scalability" (page 157) for more information.) Replay load from an Oracle trace file (See "Replay Load from an Oracle Trace File" (page 160) for more information.) Test Stored Procedures for Scalability (See "Test Stored Procedures for Scalability" (page 187) for more information.) Replay load from SQL server trace table (See "Replay Load from SQL Server 2000/2005 Trace Table" (page 163) for more information.) Import from Foglight Performance Analysis (See "Import from Foglight Performance Analysis Repository" (page 164) for more information. Import from a Delimited Text file (See "Import from a Delimited Text File" (page 165) for more information.) Replay load from an ODBC trace file (See "Replay Load From an ODBC Trace File" (page 167) for more information. Import from Oracle Dynamic Performance View (See "Import from Oracle Dynamic Performance View" (page 168) for more information.)

Benchmark Factory for Databases User Guide Creating Load Scenarios

124

l l

Import from SQL files (See "Import from SQL Files" (page 169) for more information.) Create a custom load scenario (Scalability, Goal...) (See "Create a Custom Load Scenario (Scalability, Goal...)" (page 170) for more information.)

6
Settings
About Settings
The Settings window displays default values used when creating new scripts. Changes to these settings affect only new scripts, not existing scripts. To Display Benchmark Factory Settings
l

Select Edit | Settings. The Settings dialog displays.

The Settings window provides the following dialogs:

l l l l l l l

General Benchmarks Timing User Load Latency Repository Agent

General Settings
The General settings dialog specifies workplace settings and error handling options.

Benchmark Factory for Databases User Guide Settings

126

Section Workplace Settings

Field Reload last script at startup

Description Loads the last saved script file when the program is launched.

Error Handling

Stop after first error

Default setting for a new jobs. Terminates the test when a server error occurs. The default location used when saving scripts. These files usually reside on a local drive, but may reside on a network file server. Location of error log files.

Directories

Script Files

Error Logs

Benchmark Factory for Databases User Guide Settings

127

Section

Field Data Files

Description Location of data files.

Retrieval Options

Allows you to limit the number of rows retrieved during SQL transaction execution.

Benchmarks Settings
The Benchmarks dialog specifies benchmark properties and options.

Section Benchmarks Benchmark Implementation

Options/Icons

Description Show Properties button. Displays benchmark properties. Show Benchmark DLL Properties icon. Displays DLL information.

Benchmark Factory for Databases User Guide Settings

128

Timing Settings
The Timing dialog specifies timing phases associated with an item.

Note: Setting the timing phase sets the default value for a new load scenario.

Section Time Phase

Field Total Iteration Length

Property Definition The composite of Pre-Sampling and Sampling time. The length of time users execute transactions in order to reach a steady state

Pre-Sampling

Benchmark Factory for Databases User Guide Settings

129

Section

Field

Property Definition before statistics are collected.

Sampling

The length of time to collect statistics while the users are executing transactions. Begins all users immediately when the test begins. Starts the users at the set interval determined they the set time divided by the number of users to start. Starts a new user at the set interval.

User Startup

Start all users as quickly as possible Start all users at even intervals for [time]

Start a new user every [n] seconds

User Load (Measurement Intervals)

The User Load dialog specifies the number of virtual users for each iteration of a test.

Benchmark Factory for Databases User Guide Settings

130

Note: Setting the user load sets the default value for a new load scenario. Section User Load(s) Field/Icon Description Determines user load to add to the selected list.

Add Single User Load

Adds the specified user load to add to Selected User Load(s).

Add a Range of User Loads

Adds a range of user loads to selected user load(s).

Benchmark Factory for Databases User Guide Settings

131

Section

Field/Icon Start

Description Starting user load.

End

Ending user load.

Interval

Interval of the user load.

Adds a user load to the Selected Users Load(s)

Selected User Load(s)

Lists the selected user loads.

Removes the selected user load.

Removes all user loads.

LatencySettings
The Latency settings dialog specifies the speed in which transactions are sent to the server to model real world user interactions.

Benchmark Factory for Databases User Guide Settings

132

Note:Setting the latency sets the default value for a new load scenario. Latency Type: No Delay Interarrival Time Keying Time Think Time Distribution Models Absolute Uniform Negative Exponential Normal Poisson

Benchmark Factory for Databases User Guide Settings

133

Repository Settings
Note: If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark Factory will not work against this repository. The repository is a database where all of the test results are stored. Benchmark Factory inserts test results into the repository and provides an easy way to access all test results data. By default, the Repository is a MySQL database that resides on the same machine as Benchmark Factory for Databases. The Repository can reside on another database server if required, to change the database select the Data Source Name of the ODBC connection for the new database. Repository settings allows you to edit DSN, perform ODBC administration, and test the connection. Benchmark Factory also provides a Repository Manager and Data Repository Migration wizard to assist you with repository management. Note: If the database structure does not exist on the selected database, a prompt to create the structure will appear when OK is clicked. The supported database types are:
l l l l

MySQL SQL Server Sybase Adaptive Server Anywhere (iAnywhere) Oracle

Benchmark Factory for Databases User Guide Settings

134

Section Connection Parameters

Field/Icon Data Source Name

Description Data Source name of the ODBC connection used to connect to the repository database.

User Name

The User Name used to log into the selected database.

Password

The Password associated with the user name used to log into the database.

Displays the ODBC connection information dialog for the selected data source.

Displays the ODBC Data Source Administrator dialog, used to add and edit ODBC connections.

Tests the connection of the currently selected ODBC Data Source.

Agent Settings
The Agent Settings tab allows you to manage Agent startup, shutdown, and maximum virtual user per agent settings.

Benchmark Factory for Databases User Guide Settings

135

Section Agent Management

Field Start agents automatically

Description Automatically starts Agent when Benchmark Factory begins to run a test.

Shutdown agents automatically

Automatically shuts down Agent after Benchmark Factory finishes running a test.

Maximum virtual users per agent:

Allows you to set the maximum number of virtual users per agent. For example, if you are running a load scenario of 1200, and the maximum virtual user per agent is 400, three agents will automatically start.

7
Drivers
Driver Selection
Note: Controls with yellow background can be scripted. Drivers connect Benchmark Factory to the respective database. Benchmark Factory provides the following database drivers:
l l l l l l l

IBM DB2CLI Microsoft SQL Server (Native) Microsoft SQL Server (ODBC) MySQL (ODBC) ODBC Oracle Sybase

Adding BFScript to the Profile Page

You can add BFScripts to the Benchmark Factory Data Source Names, allowing you to have individual agents test against a different server. BFScripts have been enabled on the User name and Password fields. To add BFScripts to the Data Source Name 1. Click inside Data Source Name. 2. Type the desired script name.

Benchmark Factory for Databases User Guide Drivers

137

IBM DB2CLI
The IBM DB2CLI driver requires the following connection parameters:

Note: BFScripts have been enabled on the User name and Password fields.

Database Alias:The name of the database. User Name:The name of the user associated with the server. Password:The password associated with the user.

Benchmark Factory for Databases User Guide Drivers

138

Microsoft SQL Server (Native)

Note: When using SQL Server 2005, the Microsoft SQL Server (ODBC) driver is the preferred driver. Microsoft is no longer installing DB-Library for database client connectivity. The Microsoft SQL Server driver requires the following connection parameters:

Note: BFScripts have been enabled on the User Name and Password fields. Server Name:The name or the IP address of the Microsoft SQL server. Database:The name of the database to be tested. User Name:The name of the user associated with the server. Password:The password associated with the user.

Note: The Microsoft SQL Server driver features a trusted security option. Checking the Use trusted security check box allows you to create a profile without entering a User Name or Password.

Microsoft SQL Server (ODBC)

Note: When using SQL Server 2005, the Microsoft SQL Server (ODBC) driver is the preferred driver. Microsoft is no longer installing DB-Library for database client connectivity. The Microsoft SQL Server (ODBC) driver requires the following connection parameters:

Note: BFScripts have been added to the Data Source name field on the ODBC Profile page. BFScripts have been enabled on the User Name and Password fields.
l

Data Source Name: The name of the MS SQL Server ODBC data source. User Name: The name of the user associated with the server. Password: The password associated with the user.

Benchmark Factory for Databases User Guide Drivers

139

MySQL (ODBC)
The MySQL ODBC Server driver requires the following connection parameters:

Note: BFScripts have been added to the Data Source name field on the ODBC Profile page. BFScripts have been enabled on the User Name and Password fields.
l

Data Source Name:The name of the MySQL ODBC data source. User Name:The name of the user associated with the server. Password:The password associated with the user.

MySQL ODBC
The MySQL ODBC Server driver requires the following connection parameters:

Note: BFScripts have been added to the Data Source name field on the ODBC Profile page. BFScripts have been enabled on the User Name and Password fields.
l

Data Source Name:The name of the ODBC data source. User Name:The user name associated with the server. Password:The password associated with the user.

Oracle
The Oracle driver requires the following connection parameters:

Benchmark Factory for Databases User Guide Drivers

140

Note: BFScripts have been enabled on the User name and Password fields. User/Schema: System under test you are connection to. Password: Password for system you are connecting to. TNS Tab: Allows you to connect to a database using the TNS names. Direct Tab: Allows you to connect to a database using Host, Port, Server name or SID. Connect as: Type of connection you connect to the database with: Normal, SYSDBA, or SYSOPER. Connect Using: Provides a dropdown to select the data base type (for example, Oracle XE or 11g) to connect to. Make this the BMF default home: Sets this profile connection as the default. Verify connection before proceeding: Verifies the profile is properly connected to the database before clicking Next.

Sybase
The Sybase driver requires the following connection parameters: Note: BFScripts have been enabled on the User Name and Password fields.

Server Name:The name or the IP address of the Sybase server. Database:The name of the Sybase server database. User Name:The name of the user associated with the server. Password:The password associated with the user.

Benchmark Factory for Databases User Guide Drivers

141

8
Profiles
Overview
Profiles are sets of information that Benchmark Factory uses to communicate with the systemunder-test. This information may include, server name, IP address, and user name. You create profiles using the Profile Creation Wizard.

Edit Profiles
Benchmark Factory Profiles can be created, deleted, and changed to meet script requirements.

Click here for procedures on editing profiles.

Benchmark Factory for Databases User Guide Profiles

143

Buttons

Description Creates a new profile using the Profile Creation Wizard. Imports an existing profile from the Repository The database where all Benchmark Factory information is saved. It contains information about the transactions stored in each benchmark DLL and is where all of the statistical information is stored about benchmark executions..

Deletes the highlighted profile

Displays the profile properties.

Import Profiles
Profiles contains information required to connect to the system-under-test, such as server name, IP address, and user name. Benchmark Factory allows you to import profiles from the Repository The database where all Benchmark Factory information is saved. It contains information about the transactions stored in each benchmark DLL and is where all of the statistical information is stored about benchmark executions.. Click here for procedures on how to import Profiles.

Profile Properties
Overview
Profile properties provides the following tabs:
l l l

General Options Real-Time Counters

Benchmark Factory for Databases User Guide Profiles

144

l l

Statistics Tab Clustering (Oracle only)

General Tab
The General Tab allows you to enter database parameters, installed client connection parameters, and test profiles for connection.

Section Parameters

Field/Icon User/Schema

Description Name of the user and schema for the database you are logging into. Password for the databse. Transparent Network Substrate name allowing the database to

Password TNS

Benchmark Factory for Databases User Guide Profiles

145

Section

Field/Icon

Description communicate across a network.

Direct Connect as: Alllows you to connect as the system database administor, Allows you to select the type of client you are connecting to. Clicking this checkbox defaults Benchmark Factory to this client. Confirms a good connection between Benchmark Factory and the database client.

Installed Clients

Connect Using:

Make this the BMF default home Test Profile Test Profile

Options Tab
The Profile Options tab allows you to adjust timeout and reconnect.

Benchmark Factory for Databases User Guide Profiles

146

Section Timeout

Field/Icon Time

Description The maximum amount of time Benchmark Factory will try to log on to the system-under-test. If this amount of time is reached, Benchmark Factory will return an error. Prevents the logon to the systemunder-test from timing out and returning an error. Enables Benchmark Factory to attempt to reconnect to the systemunder-test if the connection is lost. The number of times to attempt to reconnect before aborting. How long to wait before attempting to reconnect.

Infinite Timeout

Reconnect

Enable Reconnect

Number of reconnect attempts

Time to wait between reconnect attempts (seconds)

Profile Properties Real-Time Counters Tab

Real-time counters keep track of statistics during the running of a job. By default the following real-time counters are recorded when running a job even though they are not displayed:
l l l l l l l l l l

AVG_TIME BPS DEADLOCKS TOTAL_ERRORS MAX_TIME RPS TOTAL_BYTES TOTAL_ROWS TPS USERLOAD

Benchmark Factory for Databases User Guide Profiles

147

The Real-Time Counters tab displays on the profile and job property pages. The combination of these settings are used to create a complete list of real-time counters that are recorded during test execution. From this tab you can:

1. Add real-time counters to the profile 2. Delete real-time counters from the profile To Add Real-Time Counters 1. Click the Add New Counter button dialog displays. . The Browse Performance Counters

Benchmark Factory for Databases User Guide Profiles

148

2. Click the Select counters from list radio button. 3. Select the desired counter(s). 4. Click Add. 5. Click Close. To delete Real-Time counters 1. From the Real-Time Counters tab Select the counter you wish to delete.

Benchmark Factory for Databases User Guide Profiles

149

2. Click the Delete the highlighted counter button

Clustering Tab
Clustering is the process of using two or more computer systems that work together. Multiple servers are linked to handle variable workloads or to provide continued operation in the event one fails. Computers may be multiprocessor systems. A cluster of four computers with four CPUs each, provides a total of sixteen CPUs processing at one time. The Clustering tab in Benchmark Factory allows you to perform Oracle Real Application (RAC)Testing.

Click here for procedures on how to conduct Oracle RAC testing.

9
Wizards
Overview
Benchmark Factory wizards streamline the load testing processing and provide the flexibility to customize testing workloads. The following wizards are provided:
l l l l l l l l l

Benchmark Objects Data Repository Migration Wizard Execute File Load Scenario Wizard Profile Creation BFScript Submit Job Transaction User Scenario

Benchmark Objects Wizard

Benchmark Objects Wizard
The Benchmark Objects Wizard creates or clears the database objects (tables/indexes, etc.) needed to run industry standard benchmarks. The Benchmark Objects Wizard is able to run if you have a root node job, load scenario, Create/Delete Benchmark object, or Execute File node selected. The Create/Delete Benchmark object node will be added just below the item selected or as a child of the job. The Benchmark Objects Wizard will:
l l l

Select the benchmark to create objects for Determine the scale of a benchmark Create an instruction to create the benchmark objects

Benchmark Factory for Databases User Guide Wizards

151

11. Click Edit Options. The Mapping Options dialog displays. 12. Select the desired options. 13. Click OK. 14. To save the mapping options, click Save. The Save Options File As dialog display. 15. Enter the desired name. 16. Click Save. 17. Click OK.

Benchmark Factory for Databases User Guide Wizards

152

18. Click Next. The Create/Delete benchmark objects dialog displays and provides the following options:
l

Delete Benchmark Objects: Deletes existing benchmark objects (tables, indexes, etc.).

19. Select the desired option. 20. Click Next. The Benchmark Objects Wizard completion dialog displays. 21. Click Finish. The Benchmark Object node creates and displays in the Script View.

Data Repository Migration Wizard

Data Repository Migration Wizard
Overview The Data Repository Migration Wizard allows you to migrate the Benchmark Factory testing repository from one database type to another. For example, from a MySQL database to a Microsoft SQL Server database. To use the Data Repository Migration Wizard 1. Click the Start button from the Windows task bar. 2. Select All Programs | Quest Software | Benchmark Factory for Databases | Repository | Repository Data Manager. (Go to step 3). OR 1. Select All Programs|Quest Software|Benchmark Factory for Database Repository Repository Manager. The Repository Manager displays. 2. Click Data Migration. The Data Migration Wizard displays. (Go to step 3)

Benchmark Factory for Databases User Guide Wizards

153

3. Click Next. The Choose a Data Source dialog displays. 4. From the Data Source Name drop-down, select the datasource type to migrate from. 5. Enter the user name and password. 6. Click Next. The Choose a Destination dialog displays. 7. From the Data Source Name drop-down, select the datasource type to migrate to. 8. Click Next. The Data Migration Wizard completion dialog displays. 9. Click Finish.

Execute File Wizard

Execute File Wizard
Overview During the running of a workload, you may want to add an executable file (.bat or .exe) as part of a job. This gives you the flexibility of setting up your database before running a workload. For example, you could run a .bat file to set up a prepare SQL statement before running a workload. To execute a file 1. Select the Job or Load Scenario node. 2. Select Wizards | Execute File. 3. Click Next . The Execute File dialog displays. 4. Enter the path or navigate to the desired file. 5. Click the Enforce Timeout checkbox if desired. Enforce Timeout allows you to enter the desired number of seconds Benchmark Factory waits for the execution to complete before continuing the job. 6. Enter the desired number of seconds. 7. Click Next. The completion dialog displays. 8. Click Finish. The Execute File is created and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

154

Load Scenario Wizard

Load Scenario Wizard
Overview Note: To create load scenarios, you must be in the Script View. The Load Scenario Wizard is a starting point for creating Benchmark Factory scripts and walks you through the steps to build a complete workload testing script. Note: Only one Load Scenario can be run at a time. From this wizard you can test:
l

Benchmark Factory for Databases User Guide Wizards

155

Create a custom load scenario (Scalability, Goal...) (See "Create a Custom Load Scenario (Scalability, Goal...)" (page 170) for more information.)

Industry Standard Benchmark (TPC-C, TPC-H...)

This option creates load scenarios from industry standard benchmarks (AS3AP, Scalable Hardware, TPC-C, TPC-B, TPC-D, and TPC-H). This includes the steps to create/load all the required benchmark objects to execute the standard benchmark workloads. These synthetic workloads can be used when no real-world data is available to be imported for database load testing. To create an Industry Standard Benchmark script 1. Select Wizards | Load Scenario. or Click New. The Load Scenario Wizard welcome dialog displays. 2. Click Next. The Define load scenario dialog display. 3. Click Industry standard benchmark (TPC-C, TPC-H...). 4. Click Next. The Select Benchmark dialog displays. 5. Select the desired benchmark: AS3AP, Scalable Hardware, TPC-B, TPC-C, TPC-D, and TPC-H. 6. Click Next. The Profile Selection dialog displays. 7. From the Profile drop-down, select the desired profile or click New display the Profile Creation Wizard and create a profile. 8. Click Next. The Benchmark Scale dialog displays. 9. Select the desired Benchmark Scale. 10. Click the Advanced Creation Option button to map individual benchmark objects. 11. Click OK on the Advanced Creation Option page. 12. Click Next. The Measurement Intervals dialog displays. 13. Select the desired user load. 14. Click Next. The Enter Script Name dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 15. Select the desired option. 16. Click Next. The completion dialog displays.

Benchmark Factory for Databases User Guide Wizards

156

17. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 18. Click Finish. The workload creates and displays in the Script View.

Validate Oracle RAC Cluster Scalability

This options creates load scenarios to validate an Oracle RAC cluster by creating a single instance through multiple instance cluster load tests using OLTP, database warehousing, and a mixed workload. This utilizes benchmark objects and transactions from the TPC-C and the TPCH benchmarks. To validate Oracle RAC cluster scalability 1. Select Wizards |Load Scenario. or Click New. The Load Scenario Wizard displays. 2. Click Next. The Define load scenario dialog display. 3. Click Validate Oracle RAC Cluster Scalability. 4. Click Next. The Profile Selection dialog displays. 5. From the Profile drop-down, select the desired profile or click New to display the Profile Creation Wizard and create a new profile. 6. Click Next. The Edit Scale dialog displays. The Edit Scale dialog allow you to set the scale and map the objects for the TPC-C and TPC-H benchmarks. 7. If you do not desired to edit either benchmark, click Next and proceed to step 23. 8. To edit the properties of a benchmark, click the Edit button for benchmark (TPC-C or TPC-H) you wish to edit. The benchmark properties dialog displays. 9. Select the desired scale in Benchmark Scale. 10. To edit the advanced properties of the benchmark, select the Advanced tab. Here you can map properties for tablespaces and datafiles, used for the creation of standard benchmark tables and indexes. 11. Check the Use advanced options for creation of benchmarks objects check box. 12. From the Database Type drop-down, select the desired database type. 13. Select the Tables or Indexes radio button. 14. Select the object(s) you wish to map. 15. Click Edit Options. The Mapping Options dialog displays. 16. Select the desired profile from Profile drop-down. 17. Select the desired mapping options. 18. Click OK.

Benchmark Factory for Databases User Guide Wizards

157

19. To save the mapping options, click Save. The Save Options File As dialog display. 20. Enter the desired name. 21. Click Save. 22. Click OK. The Edit Scale dialog displays. 23. Click Next. The Node Selection dialog displays. 24. Enter the desired number of nodes in Enter maximum number of nodes (2-100). 25. Select the desired load balancing method. 26. Click Next. The Measurement Intervals dialog displays. 27. Select the desired load user loads. 28. Click Next. The Enter Script Name dialog displays and provides the following options:
l

29. Select the desired option. 30. Click Next. The wizard completion dialog displays. 31. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 32. Click Finish. The workload creates and displays in the Script View.

Test Database Scalability

Benchmark Factory allows you to test database scalability to ensure capacity and user demands can be met. The Load Scenario Wizard walks you through this process. To test database scalability 1. Select Wizards |Load Scenario. or Click New. The Load Scenario Wizard displays. 2. Click Next. The Define load scenario dialog displays. 3. Click Test Database Scalability. 4. Click Next. The Profile Selection dialog displays. 5. From the Profile drop-down, select the desired profile or click New to display the Profile Creation Wizard and create a new profile.

Benchmark Factory for Databases User Guide Wizards

158

6. Click Next. The Edit Scale dialog displays. The Edit Scale dialog allows you to choose scales and advanced creation options for the TPC-C and TPC-H benchmarks. 7. If you do not desired to edit either benchmark, click Next and proceed to step 23. 8. To edit the properties of a benchmark, click the Edit button for benchmark you wish to edit. The benchmark properties dialog displays. 9. Select the desired scale in Benchmark Scale. 10. To edit the advanced properties of the benchmark, select the Advanced tab. The advanced creation option allows you to set advanced properties for tablespaces and datafiles, used for the creation of standard benchmark tables and indexes. 11. Check the Use advanced options for creation of benchmarks objects check box. 12. From the Database Type drop-down, select the desired database type. 13. Select the Tables or Indexes radio button. 14. Select the object(s) you wish to map. 15. Click Edit Options. The Mapping Options dialog displays. 16. Select the desired profile from the profile drop-down. 17. Select the desired mapping options. 18. Click OK. 19. To save the mapping options, click Save. The Save Options File As dialog display. 20. Enter the desired name. 21. Click Save. 22. Click OK. The Edit Scale dialog displays. 23. Click Next. The Measurement Intervals dialog displays. 24. Select the desired user load. 25. Click Next. The Enter Script Name dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 26. Select the desired option. 27. Click Next. The wizard completion dialog displays. 28. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 29. Click Finish. The workload creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

159

Test SQL For Scalability

The SQL Scalability test allows you to execute SQL statements, allowing users to spot potential issues not seen with a single execution. Users can run variations of a SQL statement in order to find the SQL that will perform the best under a load test. To create a scalability test 1. Select Wizards | Load Scenario. or Click New. The Load Scenario Wizard displays. 2. Click Next. The Define Load Scenario dialog displays. 3. Select Test for SQL Scalability. 4. Click Next. The Define Iteration Length dialog displays. 5. When running a scalability test, you have two options for determining iteration length:
l

Number of executions per iteration-Each transaction is executed by each user for a specified number of times (recommended). Execution time per iteration-Executes each transaction for the specified length of time.

7. Select the desired option. 8. You have three testing options at this point: 1. You can click Next which takes you to step 12. 2. Add a User Scenario using the User Scenario Wizard by clicking the Add User Scenario. 3. Add transaction(s) using the Transaction Wizard by clicking Add a SQL Statement. 9. Select the desired option and follow the wizard steps. 10. After completing the chosen wizard, you are taken back to the Define Iteration Length dialog. 11. Add another transaction or user scenario, or Click Next. The Measurements Interval dialog displays. 12. Select the desired virtual user loads. 13. Click Next. The Enter Script Name dialog displays and provides the following options:
l

14. Select the desired option. 15. Click Next. The completion dialog displays.

Benchmark Factory for Databases User Guide Wizards

160

16. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 17. Click Finish. The workload creates and displays in the Script View.

Replay Load from an Oracle Trace File

The SQL contained within an Oracle Trace file can be imported into Benchmark Factory. When this wizard completes, a User Scenario(s), with the timed event sequence of captured SQL Transactions, will be placed into a Replay Load Scenario (workload). Importing an Oracle Trace file allows you to replay your database activity with the same timing as was originally captured on an existing in-house Oracle database application. To create a import from an Oracle Trace File

Note: To create an Oracle trace file which includes bind parameters, a level number of 4 must be used. For example, to start an Oracle trace for a user session, use the following command: ALTER SESSION SET EVENTS '10046 trace name context forever, level 4' 1. Select Wizards | Load Scenario.

or Click New. The Load Scenario Wizard displays. 2. Click Next. The Select Load Scenario dialog displays. 3. Click Replay load from an Oracle Trace file.

Benchmark Factory for Databases User Guide Wizards

161

4. Click Next. The Oracle 8i/9i/10g/11g Trace Input dialog displays. 5. Click on the Add Trace button. The Select File Location window displays. 6. Navigate to and select the appropriate file(s). 7. Click Open. The trace file displays under the Trace File Name column. 8. Repeat steps 4 6 until all trace files are added. 9. Click Next. The Oracle Trace Activity window displays.

10. Click the appropriate Date Range radio button. 11. If appropriate, select the "From" and "To" dates and times. 12. Uncheck the Include system activity (SYS USER) box to include system activity. Note: It is recommended to not check the Include system activity (SYS USER) check box. This imports Oracle SQL statements routinely run by Oracle for maintenance reasons and is not required for the load testing process. 13. Click Next. 14. One of the following actions takes place: 1. If the trace import file exceeds the Benchmark Factory limit for displaying individual transaction, the following dialog displays:

Benchmark Factory for Databases User Guide Wizards

162

If you click Yes, the trace import file import continues. Individual SQL will be converted to XML files. You can then edited the XML files in the Benchmark Factory Session Editor. Click the desired option. b. If the trace import does not exceed the import limit for displaying individual transactions, this dialog will not display. 15. The Enter Script Name dialog displays and provides the following options: 16. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 17. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 16. Select the desired option. 17. Click Next. The wizard completion dialog displays. 18. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 19. Click Finish. The workload creates and displays in the Script View.

Test Stored Procedures for Scalability

Stored procedures enables developers who write stored procedures in Oracle and SQL Server to test performance and scalability of the logic under varying user load levels. Benchmark Factory allows you to import SQL from database stored procedures into a Mix Load Scenario (workload). This gives you the ability to import SQL transactions, from stored procedures, for further testing under load. To test stored procedure for scalability

1. Select Wizards |Load Scenario. or Click New. The Define load scenario dialog displays. 2. Click Next. 3. Click Test Stored Procedure for Scalability.

Benchmark Factory for Databases User Guide Wizards

163

4. Click Next. The Profile Selection dialog displays. 5. From the Profile drop-down, select the desired profile. 6. Click Next. The Select Stored Procedures dialog displays. (Click here for stored procedure examples.) 7. Check the desired procedures. 8. Click Next. If the selected procedures contain bind parameters, the Bind Parameters dialog displays. 9. From SQL Name, select the desired SQL statement. 10. To add a parameter, right-click on the parameter. A drop-down displays. 11. Click Add. 12. Edit the bind parameter as needed. 13. Click Next. The Measurement Intervals dialog displays. 14. Select the desired user loads. 15. Click Next. The Enter Script Name dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 16. Select the desired option. 17. Click Next. The wizard completion dialog displays. 18. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 19. Click Finish. The workload creates and displays in the Script View.

Replay Load from SQL Server 2000/2005 Trace Table

Note: Please refer to:Creating a SQL 2000 Trace Table Using the SQL Server Profiler or Creating a SQL 2005 Trace Table Using the SQL Server Profilerfor procedures on creating trace files. The SQL contained within a SQL Server 2000/2005 trace table, generated by SQL Profiler 2000 or 2005, can be imported into Benchmark Factory. When the Load Scenario Wizard completes a User Scenario, with the timed event sequence of captured SQL Transactions, it is placed into a Replay Load Scenario (workload). Importing a SQL Server 2000/2005 trace table allows you to replay your database activity with the same timing as was originally captured on an existing inhouse SQL Server 2000/2005 database application. Note: To import a trace file, you will need to use SQL profiler to save the file as a trace table and then import the table.

Benchmark Factory for Databases User Guide Wizards

164

To replay a load from SQL Server 2000/2005 1. Select Wizards |Load Scenario. or Click New. The Load Scenario Wizard displays. 2. Click Next. The Select Load Scenario dialog displays. 3. Click Replay load from SQL Server trace table. 4. Click Next. The SQL Server Authentication window displays. 5. Select a host from the Select a host drop-down list or type in the host. 6. Select the desired connection method and fill in the required connection parameters. 7. Click Next. The SQL Server Trace Table dialog displays. 8. Select a database from the Select a Database drop-down list. 9. Select a table from the Select a Table drop-down list. 10. Click Next. The SQL Server Trace Processing window displays. 11. When the processing is complete, click Next. The Enter Script Name dialog displays and provides the following options:
l

12. Select the desired option. 13. Click Next. The completion dialog displays. 14. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 15. Click Finish. The workload creates and displays in the Script View.

Import from Foglight Performance Analysis Repository

Benchmark Factory allows you to import SQL from the Foglight Performance Analysis repository. When the Load Scenario Wizard completes, a Mixed Workload containing the SQL Transaction selected from the Performance Analysis Repository filters will be created. This gives you the ability to import a real-world workload captured from an existing in-house database application. To import from Foglight Performance Analysis Repository 1. Select Wizards |Load Scenario.

Benchmark Factory for Databases User Guide Wizards

165

or Click New. The Load Scenario Wizard displays. 2. Click Next. The Define load scenario dialog displays. 3. Click Import from Foglight Performance Analysis Repository. 4. Click Next. The Performance Analysis Service Selection dialog display. 5. Select the desired service type from the drop-down list. 6. Click Next. The Connection Information dialog displays. 7. Enter the required parameters. 8. Click Next. The Foglight Performance Analysis dialog displays. 9. Click on Time Range link and edit the Time Range and select the desired timing filters. 10. Click OK. 11. Select the desired instance view and view filters. 12. Click Next. The Measurement Intervals dialog displays. 13. Select the desired user load(s). 14. Click Next. The Enter Script dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create a and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 14. Select the desired option. 15. Click Next. The wizard completion dialog displays. 16. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 17. Click Finish. The workload creates and displays in the Script View.

Import from a Delimited Text File

The SQL contained within a delimited text file can be imported into Benchmark Factory. When this wizard completes, a User Scenario, with the ordered sequence of captured SQL Transactions, will be placed into a Replay Load Scenario (workload). Importing a delimited text files allow you to load test your database using workloads from an existing in-house generic database applications.

Benchmark Factory for Databases User Guide Wizards

166

To Import Delimited Text File 1. Select Wizards |Load Scenario.

or Click New. The Load Scenario Wizard displays. 2. Click Next. The Define load scenario dialog displays. 3. Click Import from a Delimited Text File. 4. Click Next. The Text File Input dialog displays. 5. Click Browse. 6. Navigate to and select the desired file. 7. Click Open. 8. Click Next. The Select file format dialog displays. 9. Click the desired format (Delimited or Fixed field). 1. If Fixed Field is selected, select the desired Row delimiter from the drop-down. 2. Select the desired rows to skip in Skip row. or 1. If Delimited is selected, select the desired row delimiter, text qualifier, and rows to skip. 2. Click Next. The Specify Column Delimiter dialog displays. 3. Select the desired properties. 4. Click Next. The Fixed Field Column Positions dialog displays. 5. Select the desired properties. 10. Click Next. The Enter Script Name dialog displays and provides the following options:
l

Create a new Benchmark Factory Script file-Selecting this option allows you to create a and save a completely new script. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with.

11. Select the desired option. 12. Click Next. The completion dialog displays. 13. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 14. Click Finish. The workload creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

167

Replay Load From an ODBC Trace File

The SQL contained within an ODBC trace file, generated by the ODBC Data Source Administrator, can be imported into Benchmark Factory. When the Load Scenario Wizard completes a User Scenario(s), with the ordered sequence of captured SQL Transactions, it will be placed into a Replay Load Scenario (workload). Importing an ODBC trace file allows you to load test your database using workloads from an existing in-house ODBC database application. Please click here for procedures on creating an ODBC trace file. To Import from an ODBC trace file 1. Select Wizards |Load Scenario. or Click New. The Load Scenario Wizard displays. 2. Click Next. 3. Click Replay load from an ODBC trace file. 4. Click Next. The ODBC Trace Input dialog displays. 5. Type in or browse to the ODBC trace file to import. 6. Click Next. The ODBC Trace Processing window displays. Wait for the processing to complete. 7. Click Next. The Enter Script Name dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 8. Select the desired option. 9. Click Next. The wizard completion dialog displays. 10. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 11. Click Finish. The workload creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

168

Import from Oracle Dynamic Performance View

Benchmark Factory allows you to import SQL from Oracle Dynamic Performance Views. When the Load Scenario Wizard completes, a Mixed Workload containing the most often executed, time consuming, or recently executed SQL Transaction will be created. This gives you the ability to import SQL transaction identified by the Oracle Dynamic Performance Views for further testing under a load. To import an Oracle Dynamic Performance View 1. Select Wizards |Load Scenario. or Click New. 2. Click Next. The Define load scenario dialog displays. 3. Click Import from Oracle Dynamic Performance View. 4. Click Next. The Oracle DPV Connection dialog displays. 5. Enter the appropriate connection information. 6. Click Next. The Oracle DPV Import dialog displays. 7. Click the desire radio button. 8. Enter the number of desired SQL Statements in Maximum number of SQL statements returned (100 max). 9. Uncheck the Include system activity (SYS USER) box to include system activity. Note: It is recommended to not check the Include system activity (SYS USER) check box. This imports Oracle SQL statements routinely run by Oracle for maintenance reasons, and is not required for the load testing process. 10. Click Next. The Measurement Interval dialog displays. 11. Select the desired user loads. 12. Click Next. The Enter Script dialog displays and provides the following options:
l

13. Select the desired option. 14. Click Next. The wizard completion dialog displays. 15. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 16. Click Finish. The workload creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

169

Import from SQL Files

Benchmark Factory allows you to import SQL files exported from other Quest Software products. When the Load Scenario Wizard completes, transactions containing the exported SQL will be created. This gives you the ability to import SQL transactions, identified by Quest Software products, for further testing a under a load. To import from SQL files 1. Select Wizards |Load Scenario. or Click New. The Load Scenario Wizard displays. 2. Click Next. The Define load scenario dialog displays. 3. Click Import from SQL files. 4. Click Next. The SQL Tuning File Input dialog displays. 5. Click Browse. 6. Navigate to the desired tuning file. 7. Click Open. 8. Click Next. The Measurement Intervals dialog displays. 9. Select the desired user load. 10. Click Next. The Enter Script dialog displays and provides the following options:
l

11. Select the desired option. 12. Click Next. The wizard completion dialog displays. 13. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 14. Click Finish. The workload creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

170

Create a Custom Load Scenario (Scalability, Goal...)

Custom workloads allow you to create workloads from user provided SQL. These tests provide flexibility for your load testing requirements. The following customs load scenarios are provided:
l

Mix Test-Runs a transaction mix based upon weights for a specified time at each predetermined user load level. Replay Test-Runs multiple transactions with each one running independently on a specified number of users. Goal Test-Used to find maximum throughput or response time values. A transaction mix is executed at user load levels, determined by setting a beginning, ending, and interval value. Scalability Test-Compares the performance of SQL Statement variations under a workload. Each transaction will execute individually for each user load and timing period.

Mix Test
A Mixed Workload test runs for a specified time at each predetermined user load level. Each user will run a transaction mix based upon the weights defined on the transactions tab. For example, if a test has two transactions, A and B, with A having a weight of one and B having a weight of four, on average B will run four times for every time A is run once. The run order will be randomly generated for each user so they are not all running the same transaction simultaneously. That run order is used for that user each time the test is performed to ensure reproducible results. To create a mix test 1. Select Wizards |Load Scenario.

Or Click New. The Load Scenario Wizard displays. 2. Click Next. The Define load scenario dialog displays. 3. Click Create a custom load scenario (Scalability, Goal, ...). 4. Click Next. The Test Type dialog displays. 5. Click the Mix radio button. 6. Click Next. The Define Transaction Mix dialog displays. You have three testing options at this point:

Benchmark Factory for Databases User Guide Wizards

171

1. You can click Next and create a blank mix test, and proceed to step 10. 2. Add a User Scenario using the User Scenario Wizard by clicking the Add User Scenario button. 3. Add transaction(s) using the Transaction Wizard by clicking Add a SQL Statement. 7. Select the desired option and follow the wizard steps. 8. After completing the chosen wizard, you are taken back to the Define Transaction Mix dialog. 9. Add another workload, or Click Next. The Measurements Interval dialog displays. 10. Select the desired virtual user load. 11. Click Next. The Enter Script Name dialog displays and provides the following options:
l

12. Select the desired option. 13. Click Next. The wizard completion dialog displays. 14. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 15. Click Finish. The workload creates and displays in the Script View.

Replay Test
A Replay Test runs multiple transactions with each one running independently on a specified number of users. The test will run until the defined number of executions for each transaction or a specified time limit is reached. To create a replay test 1. Select Wizard | Load Scenario.

Benchmark Factory for Databases User Guide Wizards

172

6. Click Next. The Define Replay Items dialog displays. 7. Check the Execute by time check box if you want the test to run a specified time. 8. You have three testing options at this point: 1. You can click Next and create a blank replay test, and proceed to step 11. 2. Add a User Scenario using the User Scenario Wizard by clicking the Add User Scenario button. 3. Add transaction(s) using the Transaction Wizard by clicking Add a SQL Statement. 9. Select the desired option and follow the wizard steps. 10. After completing the chosen wizard, you are taken back to the Define Replay Items dialog. 11. Add another workload, or Click Next. The Enter Script dialog displays and provides the following options:
l

Goal Test
A Goal Test uses a transaction mix A transaction is a single unit of work in the testing process, used in load scenariosand user scenarios. For example, a SQL statement. based on userload intervals. The userloads are determined by setting a beginning, ending value, and interval value. This value specifies an interval to increase the number of users tested for each iteration of the Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration and the test ends once the goal or maxim user load has been reached. To create a goal test 1. Select Wizards |Load Scenario. Or Click New. The Load Scenario Wizard displays.

Benchmark Factory for Databases User Guide Wizards

173

2. Click Next. The Select Load Scenario dialog displays. 3. Click Create a custom load scenario (Scalability, Goal, ...). 4. Click Next. The Test Type dialog displays. 5. Click the Goal radio button. 6. Click Next. The Goal Page dialog displays. The Goal tab provides the following options:
l

Find maximum TPS: Selecting this option will show how the maximum TPS can be reached during a load test. Find maximum userload where response time is less than [n] ms: Selecting this option shows how a maximum userload can be reached when response time is less than a keyed in value.

7. Select the desired option. 8. Set the desired Start, Ending, and Interval in Userloads. For this test, the defaults values of 1, 5, and 1 are used. When run, User Loads of 1, 2, 3, 4, 5 are run until the TPS of one run is less than the TPS of the previous run. 9. Click Next. The Define Transaction Mix dialog displays. 10. You have three testing options at this point: 1. You can click Next and create a blank test and proceed to step 13. 2. Add a User Scenario using the User Scenario Wizard by clicking the Add User Scenario button. 3. Add transaction(s) using the Transaction Wizard by clicking Add a SQL Statement. 11. After completing the chosen wizard, you are taken back to the Define Transaction Mix dialog. 12. Add another workload, or Click Next. 13. The Enter Script dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 14. Select the desired option. 15. Click Next. The wizard completion dialog displays. 16. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 17. Click Finish. The workload creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

174

Scalability Test
The SQL Scalability test allows you to execute SQL statements, allowing users to spot potential issues not seen with a single execution. Users can run variations of a SQL statement in order to find the SQL that will perform the best under a load test. To create a scalability test 1. Select Wizards | Load Scenario. or Click New. The Load Scenario Wizard displays. 2. Click Next. The Select Load Scenario dialog displays. 3. Click Create a custom load scenario (Scalability, Goal, ...). 4. Click Next. 5. Click the Scalability radio button. 6. Click Next. The Define Iteration Length dialog displays. 7. When running a scalability test, you have two options during the running of the workload:
l

8. Select the desired option. 9. You have three testing options at this point: 1. You can click Next and create a blank scalability test and proceed to Step 12. 2. Add a User Scenario using the User Scenario Wizard by clicking the Add User Scenario button. 3. Add transaction(s) using the Transaction Wizard by clicking Add a SQL Statement. 10. Select the desired option and follow the wizard steps. 11. After completing the chosen wizard, you are taken back to the Define Transaction Mix dialog. 12. Add another workload, or Click Next. The Measurements Interval dialog displays. 13. Select the desired virtual user load(s). 14. Click Next. The Enter Script dialog displays and provides the following options:

Benchmark Factory for Databases User Guide Wizards

175

15. Select the desired option. 16. Click Next. The wizard completion dialog displays. 17. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 18. Click Finish. The workload creates and displays in the Script View.

Profile Creation Wizard

Profile Creation Wizard
Overview Profiles are used by Benchmark Factory to connect to the system-under-test. The profile contains information required to connect to the system-under-test, such as server name, IP address, and user name. The Profile Creation Wizard walks you through steps required to create a Benchmark Factory profile. To create a profile 1. Select Wizards | Profile Creation. 2. Click Next. 3. From the Driver drop-down list, select the required driver type. 4. Click Next. 5. Enter the User/Schema. 6. Enter the Password. 7. From the TNSDriver drop-down, select the required database. or Click the Direct tab and enter the required fields. 8. From the Connect as drop, select the required connection, Normal, SYSDBA, or SYSOPER. 9. Select the installed client if applicable.

Benchmark Factory for Databases User Guide Wizards

176

10. Click the Verify connection before proceeding check box to verify the connection to the specified database.

11. Click Next. 12. Enter a profile name. 13. Click Finish.

BFScript
BFScript Wizard
Note: Controls with yellow background can be scripted. Benchmark Factory scripting allows you to randomize the load testing process. Scripting can be utilized by selecting the SQL Statement node and inserting the script in the properties view. A field has scripting capabilities if its right-click menu contains the Script Wizard option. The Script Wizard is a quick and easy way to use Benchmark Factory scripts. The Script Wizard consists of a script category and a script function. Each script function has a short description included and if applicable, its parameters.

Benchmark Factory for Databases User Guide Wizards

177

There are two features within Benchmark Factory that assist you when using scripting capabilities:
l l

Script Wizard Script Assist

Click here for an overview of $BFScripts. Script Wizard 1. Right-click the edit control. A drop-down display.

2. Select BFScript Wizard from the drop-down. The Script Wizard displays.

Benchmark Factory for Databases User Guide Wizards

178

3. Select the desired script. 4. Some scripts require further parameters settings. For a complete overview of individual script settings, please refer to Scripts Overview. If scripts require parameters: 1. Click Next. 2. Enter the required parameters. 5. Click Finish. The script and parameters display in the Script Wizard. Script Assist 1. Type in the desired SQL statement. When $BF is typed the Script Assist is activated allowing you to insert the desired script.

Benchmark Factory for Databases User Guide Wizards

179

2. Select the desired script. 3. Press Enter. The script displays. 4. Type in parameters as required.

SQL Scalability Wizard

SQL Scalability Wizard
Note: To run the SQL Scalability Wizard from Toad you must have Benchmark Factory loaded. The Benchmark Factory console does not have to be open to run the SQL Scalability Wizard. Overview The SQL Scalability test allows you to execute SQL statements, letting users spot potential issues not seen with a single execution. Users can run variations of a SQL statement generated by SQL Tuning in order to find the SQL that will perform the best under a load test. Creating a SQL Scalability load scenario requires the following steps. 1. Creating the SQL tuning connection 2. Entering the desired SQL statement 3. Running the statement using the in Benchmark Factory SQL Scalability Wizard Creating the SQL tuning connection To create the SQL Turning connection 1. In Quest Central or Toad, connect to an Oracle or SQL Server database. 2. From the Quest Central main toolbar select Tools|SQL Tuning to launch. Entering the desired SQL statement To enter the desired SQL statement 1. Enter the desired SQL statement in SQL Text. 2. Click the Optimize Statement icon. Clicking this icon executes multiple syntax transformation rules to produce a list of semantically equivalent SQL statements. Running the SQL statement using the in Benchmark Factory SQL Scalability Wizard 1. Click the Benchmark Factory drop-down icon and select the desired option. Three options are provided: 2. Test for Scalability-Tests the currently displayed SQL.

Benchmark Factory for Databases User Guide Wizards

180

Benchmark Factory for Databases User Guide Wizards

181

Transactions The Transactions view provides the following statistics on the SQL statements being run:
l l l l l l l

Executions Rows Bytes Errors Average Transaction Time Minimum Transaction Time Maximum Transaction Time

Benchmark Factory for Databases User Guide Wizards

182

Benchmark Factory for Databases User Guide Wizards

183

Summary The Summary graph provides real-time and raw data summary graphs. The Summary graph displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows you to change graph settings and view.

Messages The Output window displays messages about the job being run.

Benchmark Factory for Databases User Guide Wizards

184

2. Run-Runs the test. 3. Stop-Stops a test. 4. Benchmark Factory-Launches the Benchmark Factory for Databases console. 5. Run Reports-Launches Benchmark Factory Reports.

Submit Job Dialog

Submit Job Dialog
A job contains load scenario, execute file, creation/deletion of benchmark objects, or a combination of each. The Submit Job Wizard walks you through the process of adding a job to the jobs queue. See "Jobs Queue View" (page 39) for more information. To submit a job to the Jobs Queue 1. Select the Job node. 2. Right-click.

3. Select Submit Job. The Submit Job dialog displays.

Benchmark Factory for Databases User Guide Wizards

185

The Submit Job dialog allows you to tailor your job needs and provides the following options: Section Profile Description Allows you to select profiles already created.

Displays the Profile Creation Wizard. See "Profile Creation Wizard" (page 175) for more information.

Displays the Profile setting dialog. See "General Tab" (page 144) for more information.

Actions

Benchmark Factory for Databases User Guide Wizards

186

Section

Description

Comments

Allows you to add a comment to the job.

Alerts

Allows you to send an email, pager, or net-send operator alerts when the job completes. Allows you to add Job Counters to the job.

Job Counters

Stop job after first error

When checked, the job will stop if an error is encountered.

4. After filling in the desired parameters, click Submit.

Transaction Wizard
Transaction Wizard
A transaction is a single unit of work in the testing process, used in load scenarios and user scenarios. For example, a SQL statement. The Transaction Wizard walks you through the steps required to create a transaction. From the Transaction Wizard you can:

Benchmark Factory for Databases User Guide Wizards

187

l l l l l

Test Stored Procedures for Scalability Create SQL Statement Transactions Create Prepare SQL Statement Transaction Create Execute prepared SQL Statement Transaction Import Transactions

Test Stored Procedures for Scalability

1. Select Wizards |Load Scenario. or Click New. The Define load scenario dialog displays. 2. Click Next. 3. Click Test Stored Procedure for Scalability. 4. Click Next. The Profile Selection dialog displays. 5. From the Profile drop-down, select the desired profile. 6. Click Next. The Select Stored Procedures dialog displays. (Click here for stored procedure examples.) 7. Check the desired procedures. 8. Click Next. If the selected procedures contain bind parameters, the Bind Parameters dialog displays. 9. From SQL Name, select the desired SQL statement. 10. To add a parameter, right-click on the parameter. A drop-down displays. 11. Click Add. 12. Edit the bind parameter as needed. 13. Click Next. The Measurement Intervals dialog displays. 14. Select the desired user loads. 15. Click Next. The Enter Script Name dialog displays and provides the following options:

Benchmark Factory for Databases User Guide Wizards

188

2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 16. Select the desired option. 17. Click Next. The wizard completion dialog displays. 18. If the Do you want the job to be automatically submitted to the jobs queue? checkbox displays, then checking this will cause the Submit Job dialog to display when this wizard completes. 19. Click Finish. The workload creates and displays in the Script View.

Create SQL Statement Transaction

Create SQL statement transaction allows you to add a SQL statement to your workload. To import a SQL statement transaction 1. Select a load scenario, user scenario, or transaction. 2. Click Create SQL Statement Transaction. 3. Click Next. the SQL Statement dialog displays. 4. Enter the desired SQL Statement. or Click the Script Wizard icon to Launch the BFScript Wizard.

5. After enter the desired SQL statement or completing the BFScript Wizard, Click Next. The Real World Latencies dialog displays. 6. Select the desired latency type. 7. Enter the desired mean and value time. 8. Click Next. The wizard completion dialog displays. 9. Click Finish. The SQL statement creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

189

Create Prepared SQL Statement Transaction(s)

Prepared SQL statements can be executed during a load test by using an execute prepared SQL Statement transaction. Bind variables can be added to prepared SQL statement transactions. Execution of Prepared SQL Statements In order to accommodate different testing needs, Prepared SQL Statements Transactions have the ability to Prepare and/or execute the SQL. The execution of the transaction will differ depending on the implementation method. The following provides an explanation of implementation methods when running Prepared SQL Statements in Benchmark Factory.
l

When a Prepare SQL Statement transaction is contained within a User Scenario, it will always prepare the SQL and, if Execute after prepare is checked (Figure 1) , it will execute the SQL immediately after the preparation. If the transaction is not within a User Scenario and Execute after prepare is not checked, it will always prepare the SQL without executing. If the transaction is not within a User Scenario and Execute after prepare is checked, the statement will be prepared once for each user and immediately executed, and then executed without a prepare each time after that point. This enables a Load Scenario to be optimized for better real world simulation.

Figure 1-Execute after prepare checkbox

Benchmark Factory for Databases User Guide Wizards

190

The following procedure provides examples of:

l l

Creating a prepared SQL statement transaction Creating a prepared SQL statement transaction with a bind variable

To create a prepared SQL statement transaction 1. Click the Load Scenario or User Scenario node. 2. Select Wizards|Transaction. The Transaction Wizard displays. 3. Click Next. The Define Transaction dialog displays. 4. Click Create Prepare SQL Statement Transaction. 5. Click Next. The SQL Statement dialog displays. 6. Enter the desired SQL Statement. or Click the Script Wizard icon to Launch the BFScript Wizard.

7. After enter the desired SQL statement or completing the BFScript Wizard, Click Next. The Real World Latencies dialog displays. 8. Select the desired latency type. 9. Select the desired latency type. 10. Enter the desired mean and value time. 11. Click Next. The Transaction Wizard completion dialog displays. 12. Click Finish. The transaction creates and displays in the Script View. To Create a prepared SQL statement transaction with a bind variable 1. Click the Load Scenario, User Scenario, or Transaction node. 2. Select Wizards|Transaction. The Transaction Wizard displays. 3. Click Next. The Define Transaction dialog displays. 4. Click Create Prepare SQL Statement Transaction. 5. Click Next. The SQL Statement dialog displays. 6. Enter the desired SQL Statement or

Benchmark Factory for Databases User Guide Wizards

191

Click the Script Wizard icon to Launch the BFScript Wizard.

7. Right-click inside the parameter list control. A drop-down displays.

8. Select Add. 9. Right-click inside the type list control.

Benchmark Factory for Databases User Guide Wizards

192

10. Select the desired type from the list control. 11. Click inside Parameter. 12. Enter the desired Parameter. 13. Click inside of Value. 14. Enter the desired value. 15. Enter other bind variables as required.

16. Click Next. The Real World Latencies dialog displays. 17. Select the desired latency type. 18. Enter the desired mean and value time. 19. Click Next. The Transaction Wizard completion dialog displays. 20. Click Finish. The transaction creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

193

Create Execute Prepared SQL Statement Transaction

These transactions will run the SQL statement contained with the reference prepared SQL when called. If the statement has not been prepared, an error will generate. To create SQL statement transaction 1. Click the Load Scenario or User Scenario node. 2. Select Wizards |Transaction Wizard. The Transaction Wizard displays. 3. Click Next. The Define transaction dialog displays. 4. Click Create Execute prepare SQL Statement Transaction. 5. Click Next. The SQL Statement dialog displays. 6. Enter the name of the desired Prepare SQL Statement transaction. 7. Click Next. The Real World Latencies dialog displays. 8. Select the desired latency types(s) and enter the desired times. 9. Click Next. The completion dialog displays. 10. Click Finish. The transaction creates and displays in the Script View.

Import Transactions
Import transactions allows you to import a transaction(s) from the AS3AP, Scalable Hardware, TPC-B, TPC-C, TPC-D or TPC-H benchmark standard benchmarks to build workloads. To import a transaction 1. Select Wizards | Transaction Wizard. The Transaction Wizard displays. 2. Click Next. The Define Load Transaction dialog displays. 3. Click Import Transaction. 4. Click Next. The Import Transactions dialog displays. 5. From the Benchmark drop-down list select the desired benchmark. 6. Select the desired transactions by clicking on the gray square in the first column. 7. Click Next. The Real World Latencies dialog displays. 8. Select the desired latency type. 9. Enter the desired mean or value time. 10. Click Next. The Transaction Wizard completion dialog displays. 11. Click Finish. The transaction creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

194

User Scenario Wizard

User Scenario Wizard
The User Scenario Wizard allows you to build user scenarios. A user scenario is a series of one or more transactions to be executed in order, normally associated with user behavior simulated against the system-under-test. From the User Scenario Wizard you can:
l l l l l

Import and Replay Load Scenarios From an Oracle Trace File Replay load from SQL Server trace table Import from a delimited text file Replay from an ODBC trace File Create a custom user scenario

Import and Replay Load Scenarios From an Oracle Trace File

The SQL contained within an Oracle Trace file can be imported into Benchmark Factory. When the Load Scenario Wizard completes, a User Scenario, with the timed event sequence of captured SQL Transactions, will be placed into a Replay Load Scenario (workload). Importing an Oracle Trace file allows you to replay your database activity with the same timing as was originally captured on an existing in-house Oracle database application. To import and replay an Oracle Trace File 1. Select a user scenario node and right-click. 2. Select New User Scenario. The User Scenario Wizard displays. 3. Click Next. 4. Select Replay load from an Oracle Trace file. 5. Click Next. 6. Click Add Trace. 7. Navigate to the trace file you want to import. 8. Select the file. 9. Click Open.

Benchmark Factory for Databases User Guide Wizards

195

10. Click Next. The file processes. 11. Click Next. 12. Click Finish. The user scenario(s) are created and displayed in the Script View.

Replay Load from a SQL Server Trace Table

The SQL contained within a SQL Server trace table, generated by SQL Profiler 2000 or 2005, can be imported into Benchmark Factory. When the User Scenario Wizard completes, a User Scenario, with the timed event sequence of captured SQL Transactions, it is placed into a Load Scenario (workload) in your current script. Importing a SQL Server 2000/2005 trace table allows you to replay your database activity with the same timing as was originally captured on an existing in-house SQL Server 2000/2005 database application. Note: For instructions on trace file/table creation, please see: Creating a SQL 2000 Trace Table Using the SQL Server Profiler or Creating a SQL 2005 Trace Table Using the SQL Server Profiler To Import from a SQL Server Trace Table 1. Select a load scenario, user scenario, or transaction node. 2. Click Next. 3. Select Wizards |User Scenario. The User Scenario welcome dialog displays. 4. Click Replay Load From SQL Server 2000/2005 trace table. 5. Click Next. The SQL Server 2000 Authentication window displays. 6. Select a host from the Select a host drop-down list. 7. Select the desired connection method and fill in the required connection parameters. 8. Click Next. The SQL Server 2000/2005 Trace Table dialog displays. 9. Select a database from the Select a Database drop-down list. 10. Select a table from the Select a Table drop-down list. 11. Click Next. The SQL Server 2000/2005 Trace Processing window displays. 12. Wait for the processing to complete. 13. Click Next. The wizard completion dialog displays. 14. Click Finish. The user scenario creates and displays in the Script View.

Benchmark Factory for Databases User Guide Wizards

196

Import from a Delimited Text File

The SQL contained within a Delimited Text file can be imported into Benchmark Factory. When this wizard completes, a User Scenario, with the ordered sequence of captured SQL Transactions, will be placed into a load scenario (workload) in your script. Importing a delimited text files allow you to load test your database using workloads from an existing in-house generic database applications. To Import Delimited Text File 1. Select a load scenario, user scenario, or transaction. 2. Select Wizards |User Scenario. The User Scenario welcome dialog displays. 3. Click Import from a Delimited Text File. 4. Click Next. The Text File Input dialog displays. 5. Click Browse. 6. Navigate to and select the desired file. 7. Click Open. 8. Click Next. The Select file format dialog displays. 9. Click the delimited or fixed field radio button. 10. Click Next. The Specify Column Delimiter or Fixed Field Column positions dialog displays, depending on the selection on the previous page. 11. Select the desired properties. 12. Select the desired SQL Column. 13. Click Next. The wizard completion dialog displays. 14. Click Finish. The user scenario creates and displays in the Script View.

Replay Load from an ODBC Trace File

The SQL contained within an ODBC trace file, generated by the ODBC Data Source Administrator, can be imported into Benchmark Factory. When this wizard completes a User Scenario, with the ordered sequence of captured SQL Transactions, it will be placed into a load scenario (workload) in your script. Importing an ODBC trace file allows you to load test your database using workloads from an existing in-house ODBC database application. To Import from an ODBC trace file 1. Select a Load Scenario, User Scenario, or Transaction. 2. Select Wizards|User Scenario. The User Scenario welcome dialog displays. 3. Click Next. 4. Click Replay Load from an ODBC trace file.

Benchmark Factory for Databases User Guide Wizards

197

5. Click Next. The ODBC Trace Input dialog displays. 6. Type in or browse to the ODBC trace file to import. 7. Click Open. 8. Click Next. The ODBC Trace Processing window displays. Wait for the processing to complete. 9. Click Next. The wizard completion dialog displays. 10. Click Finish. The user scenario creates and displays in the Script View.

Create a Custom User Scenario

Custom user scenarios allow you to create workloads from user provided SQL. These tests provide flexibility for your load testing requirements. To create a custom user scenario 1. Select a load scenario, user scenario, or transaction. 2. Select Wizards |User Scenario. The User Scenario Wizard welcome dialog displays. 3. Click Next. The Define user scenario dialog displays. 4. Click Create a custom user scenario. 5. Click Next. The Define User Scenario dialog displays. 6. To add a SQL transaction to your user scenario, click Add SQL Statement. The Transaction Wizard displays. 7. Click Next. The Define Transaction dialog displays. 8. Select the desired transaction type. 9. Follow the wizard procedures. 10. When the wizard completes, the transaction displays in the Define User Scenario dialog. 11. Click the Add SQL Statement button to add another transaction. or Click Next. The Enter User Scenario Name dialog displays. 12. Enter the desire name. 13. Click Next. The completion dialog displays. 14. Click Finish. The user scenario creates and displays in the Script View.

10
Testing Results (Run Reports)
Benchmark Factory TestingResults
Analysis is the ultimate goal of your load test. When you run a test, you get results both in realtime and in a saved format. Benchmark Factory provides you with testing results that are easy to interpret and allows you to attribute individual results to individual tasks and users. Whether you are viewing a real-time or historical graph, Benchmark Factory uses the same graphing tool. In most cases you will find that the graphs are presented in the most meaningful form. The Benchmark Factory graphing tool allows you to customize graphs to tailor your load testing viewing requirements. Benchmark Factory provides the following testing results types:
l l l

During the running of a workload (Real-time results) After the workload completes (Run Reports) Exported report information to Excel using the command line

Benchmark Factory Testing Results Terminology

The following provides a list of commonly used Benchmark Factory testing terms. The following terms can be expressed as: Average The average of all recorded values for the statistic over the sampling period. Minimum The minimum value the statistic obtained over the sampling period. Maximum The maximum value the statistic obtained over the sampling period. 90th Percentile This is usual associated with a timing statistic. This is the time value where 90 percent of all values recorded for a statistic fell below. Term Bytes Definition The number of bytes from data received from a SQL statement.

Bytes Per Second

The number of bytes processed per second over the sampling period. This is just the Total Bytes divided by the Sampling period in seconds.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

199

Term

Definition

Response Time

The time it takes from when the SQL is sent to the server responds.

Retrieval Time

The time it takes from the server responds to a SQL statement till the last byte of data/results is obtained.

Rows

The number of rows received from a SQL statement.

Rows Per Second (RPS)

The number of rows received per second over the sampling period. Similar to above.

Transactions Per Second (TPS )

The transactions, or SQL statements, processed by the server per second. A transaction in Benchmark Factory can be more than a single SQL statement, such as the TPC-C transaction New Order, this transaction inserts a new order by inserting one record into the new order table and 5 7 items in the orderline table.

Transaction Time (sometimes listed as just Time)

The sum of the Response and Retrieval time.

Exporting Test Results to Excel Using the Command Line

Benchmark Factory allows you to export testing results to Excel using the Command line. To export testing results from Benchmark Factory to Excel using the command line: 1. Select Start | All Programs | Accessories | Command Prompt. 2. Navigate to the Benchmark Factory for Databases bin directory.
Example: CD C:\Program Files\Quest Software\Benchmark Factory for Databases\bin

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

200

3. Enter BMFRunHistory.exe x 34 88 108. Benchmark Factory testing results with Run ids of 34, 88, and 108 export to Excel.

Real-Time Load Testing Results

Overview-Viewing Real-time Statistics Workloads
During the running of a database load test, real-time statistics provide insight into the performance of the database being tested. Numerous datapoints allow you to examine exactly how your database is performing. Benchmark Factory provides the following nodes to view realtime load testing statistics:
l l l l

Run Status Node Realtime Statistic Summary Transaction Graph

Change Graph Views

Right-clicking on a Benchmark Factory graph displays a drop-down that allows you to customize graph settings. Whether you are viewing a real-time graph or one in Benchmark Factory Run Reports, Benchmark Factory uses the same graphing tool. In most cases you will find that the graphs are presented in the most meaningful form (Line Graphs or Bar Graphs based on the data being presented.) Displays Benchmark Factory Run Reports Shows in a data form or graph form Displays the Graph Legend Displays the graph toolbar Prints the graph Copies Data to Clipboard Copies Graph to Clipboard Loads a saved graph configuration Saves a graph configuration Sets the current configuration as default Clears the current chart configuration

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

201

Show Data/Show Graph Toggling to Graph displays the data in graph form.

Graph Legend Toggling to Graph Legend displays a legend on the side of the graph.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

202

Toolbar Toggling to Toolbar displays the graph toolbar.

Print Choosing Print displays the Print Dialog. Copy Data to Clipboard Copies the data to the clipboard. Copy Graph to Clipboard Copies the graph to the clipboard. Load Configuration Benchmark factories graphs allows you to save graph configurations. Save Configuration Saves a graph configuration. Set as Default Sets a configured graph as default.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

203

Clear Configuration Clears a graph configuration.

Run Status Node

Run Status Node properties displays information on the job when it is running. You can monitor the status of the test, which userload its running, transactions per second (TPS), and a variety of other datapoints that give you insight into the actual performance your database provides under the userload being tested. Iteration Overruns Iteration overruns occur at the end of an iteration to allow time for all transactions submitted within the test iteration cycle to complete, so that all transaction statistics can be collected. For example, an agent may execute a transaction during the last five seconds of test iteration, if this transaction takes 15 seconds to complete, an iteration overrun of 10 seconds will occur. To view the Run Status node 1. Click the Run Status node in the Jobs Queue.

Real-Time Statistics Node

The Real Time Statistics node provides you with real-time graphs and raw data. This data allows you to spot system-under-test issues that may be affecting server performance. The Real Time Statistics node displays real-time counter data. Right-clicking inside the graph displays a dropdown that allows you to change graph settings and view. To view the Real-Time Statistic node 1. Click the Real-Time Statistics node in the Jobs Queue.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

204

2. Select Click here to Add Datapoints. The Add Datapoints dialog displays.

3. Select the desired datapoints to view. 4. Click OK.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

205

5. To change graph views, right-click. See "Change Graph Views" (page 311) for more information.

Summary Node
The Summary node displays user selected statistics for the job timing period following completion of an iteration. To view the Summary node 1. Click the Summary node in the Jobs Queue.

2. Click the datapoints Vs. Userload drop-down.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

206

3. Select the desired datapoints to view. 4. To change graph views, right-click. See "Change Graph Views" (page 311) for more information. 5. Select the desired option.

Transaction Node
The Transaction node displays user selected transactions statistics for the job timing period following completion of an iteration. To view the Real-time statistics node 1. Click the Summary node in the Jobs Queue. The Summary graph displays in the Properties view.

2. Click the datapoints Vs. Userload drop-down.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

207

3. Select the desired datapoints to view. 4. To change graph views, right-click. A drop-down displays. 5. Select the desired option. Click here for procedures on changing graph views.

Benchmark Factory Run Reports

Run Reports Node
Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page 212) for more information. Note: Three instances of Run Reports can be viewed at one time. Cutting and Pasting of Run Reports From Benchmark Factory run reports you can cut and paste test results into the Clipboard. See Cut and Pasting Reports for more information. Testing Server Throughput Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

208

l l l l l l l l

Transaction/Second (TPS) Transaction Time Bytes/Second (BPS) Rows/Second (RPS) Total Bytes Total Errors Total Rows Response Time

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

209

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

210

Change Graph Views

Show Data/Show Graph Toggling to Graph displays the data in graph form.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

211

Graph Legend Toggling to Graph Legend displays a legend on the side of the graph.

Toolbar Toggling to Toolbar displays the graph toolbar.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

212

Run Reports Viewer

You can view a list of all Benchmark Factory Run Reports outside or inside of Benchmark Factory Accessing a List of all Benchmark Factory Run Reports from Outside of Benchmark Factory To access a list of all of Benchmark Factory Run Reports from outside of the Benchmark Factory console 1. Click the Start button from the Windows task bar. 2. Select Programs | Quest Software | Benchmark Factory for Database |Run Reports. Run Reports displays.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

213

Accessing a List of All Benchmark Factory Jobs from Inside of Benchmark Factory To access all Benchmark Factory Jobs from inside of Benchmark Factory 1. From the Benchmark Factory console click Tools | Run Reports. Run Reports displays. Note: For procedures on viewing individual reports, please see Viewing Load Testing Results.

View Load Testing Results

Benchmark Factory Run Reports allows you to:
l l

View a single run report View multiple run reports

To view a single run report 1. From the Jobs or Scripts view select Run Reports for the desired script.

2. Select the desired run.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

214

3. Click the Show Test Results icon . Run Reports displays in the upper right-hand corner of Run Reports. Run reports displays. 4. Click here for procedures on using Benchmark Factory Run Reports.

Using Benchmark Factory Run Reports

Benchmark Factory Run Reports provides a wide array of graphs, charts, and data that allows you to a analyze load testing scripts. Run Reports provides the following nodes:
l l l l l l

Run ID Load Scenario Testbed Configuration Agent Configuration Summary Graphs Results

Run ID Node The Run ID node displays run information, profile information, and peak throughput measurements. This graph is a snapshot of the workload test. For example, the Statistics for the Userload allows you to quickly review a complete overview of database that includes, userload, transactions per second (TPS), average response time (sec).

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

215

Load Scenario Node The Load Scenario Node displays the load scenario properties that were used as the load scenario ran. This allows you to review transaction mix, timing, and user load.

Testbed Configuration Node The Testbed Configuration node shows data on agent configuration and processes during the running of a job.

Summary Graphs Nodes The Summary Graphs node provides graphs that allow you to analyze in detail the results of a workload. The following graphs types are provided:
l l l l

Transaction Comparison Cluster Performance User Load Realtime Summary

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

216

l l

Realtime Detail Distribution

Transaction Comparison Graph The Transaction Comparison graph displays when the load scenario contains multiple transaction. Individual transactions that ran during the workload test are displayed and their performance against the user load can be reviewed.

Cluster Breakdown Graph Vs User Load The cluster Breakdown graph shows you how individual Real Application Clusters (RAC) performed. By viewing individual RAC nodes, you can determine if your RAC system meets the user expectations required.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

217

User Load Graph Understanding how user loads affect the performance of a database is essential to end user satisfaction. The User Load graph plots user load against:
l l l l l l l l

Transaction/Second Transaction Times Bytes/Second Rows/Second Total Bytes Total Errors Total Rows Total Rows

Reviewing these datapoints allows you to fully understand how "real-world" userloads affect database performance.

Realtime Summary Graph Vs Userload The Realtime Statistic graph allow you to view what "actually" happened during a load test. You can plot user load against:
l l l l l

Transaction/Second Total Rows Errors Total Bytes Rows/Second

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

218

l l l l l l

Minimum Time Maximum Time Deadlocks Bytes/Second Average Time Bytes/Second

Realtime Detail Graph The Realtime Detail graph allows you to view what actually happened during the running of a load test. This allows you to view the actual timing events. From the Realtime Detail graph you can view:
l l l l l l l l l l

Transaction/Second Total Rows Errors Total Bytes Rows/Second Minimum Time Maximum Time Deadlocks Bytes/Second Average Time

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

219

Distribution Graph The Distribution Graph provides a breakdown of total transaction time and response time in percentages allowing you view how they performed in comparison to response time.

Results Node The Results Node allows you to view a variety of data on each user load run during a test. Data on individual agents can also be viewed. The Results node graph provides you with a breakdown of requested transaction mixes with actual transaction mixes for performance comparison.

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

220

View Multiple Run Results

Benchmark Factory allows you to view and compare multiple Run Reports results. To view multiple Run Reports results 1. Select the desired runs.

2. Click the Show Test Results icon Reports displays

in the upper right-hand corner of the dialog. Run

Benchmark Factory for Databases User Guide Testing Results (Run Reports)

221

3. Click Summary Graphs. The Summary Graph provides a comparison of the runs selected. Note: To change Run Reports graph views, please refer to Changing Graph Views. 4. Click the Run ID node. General Statistics displays, showing basic information about when a test was run, what profile was used when the test was run, and the peak throughput of the test. In addition, General Statistics includes the response time and transaction time statistics measured during peak throughput. 5. View the Summary Graph. 6. View other graphs as desired. 7. Close Run Reports when finished viewing run results.

Export Test Results to Excel

Benchmark Factory allows you to export test results to Microsoft Excel from: Note: Microsoft Excel must be installed to perform the following procedure. To export load test results from Run Reports 1. In Run Reports, click the Excel icon 2. Accept the default settings. 3. Click OK. Excel opens. 4. Select the desired tab and review the information. 5. Print the reports if desired. 6. Save the reports if desired 7. Select File | Exit. in the upper left-hand corner. The

11
Benchmarks
Overview
A benchmark is a performance test of hardware or software. A benchmark simulates real-world application work loads. These models of real applications can then be run against the system being evaluated. Models work better than the actual application and offers reproducible results. The real application, on the other hand, has too many variables that can change the results over several test sessions.

Realistic Expectations When Using Benchmarks

Industry standard benchmarks represent real application workloads, their run results depend on workload definition, (user load, latency, etc.) as well as server configuration and tuning parameters. During the running of a benchmark, deadlock errors can occur. This is not an issue with Benchmark Factory, but more of an issue with the tuning of the workload as well as the system-under-test. For example, it is would be unrealistic to expect no errors from a workload that runs 1000 users updating a single row on a small table. To further trouble shoot any tuning issues, we recommend Quest Software Spotlight products.

What Benchmarks Measure

Benchmarks measure performance that analyze:
l l l l l

Raw performance of a complete system Raw performance of a specific subsystem (disk, video, CPU, memory, etc.) Performance of a computer running a particular application Performance of applications running on a network Capacity of a system. This is often refereed to as capacity planning.

Provided Benchmarks
Benchmark Factory provides the following standard benchmarks:
l l l l l

AS3AP Scalable Hardware TPC-B TPC-C TPC-D

Benchmark Factory for Databases User Guide Benchmarks

223

l l

TPC-E TPC-H

AS3AP Benchmark
The AS3AP benchmark is an American National Standards Institute (ANSI) Structured Query Language (SQL) relational database benchmark. The AS3AP benchmark provides the following features:
l l l l

Tests database processing power Built-in scalability and portability that tests a broad range of database systems Minimizes effort in implementing and running benchmark tests Provides a uniform metric and straight-forward interpretation of benchmark results

Systems tested with the AS3AP benchmark must support common data types and provide a complete relational interface with basic integrity, consistency, and recovery mechanisms. The AS3AP tests systems ranging from a single-user microcomputer Database Management System (DBMS) to a high-performance parallel or distributed database.

Best Practices
Do not load-test against a production server if possible. Load-testing and benchmarking on a production server significantly degrades performance. In some cases, load-testing can cause a server to fail. However, if testing against a production server, take the following precautions:
l

Perform the testing when no other users are on the system and no automated processes are running. Users and automated processes can adversely affect testing results Have a recovery plan and backup all data prior to testing Determine how long it will take to restore a production server if it went down during load-testing Perform manual testing. Manual testing ensures that no unexpected outside activity takes place during the testing process

l l

Reinitialize the Database To reinitialize a testing database, run a job containing a Benchmark Object node. There are two ways to create a Benchmark Object node:
l

Run the Benchmark Object Wizard to add the Delete Benchmark objects for AS3AP node to a job as needed. See "Benchmark Objects Wizard" (page 150) for more information.

Benchmark Factory for Databases User Guide Benchmarks

224

Run the Load Scenario Wizard to create a new script containing the Create Objects for the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the Benchmark. Running the delete objects for 'AS3AP' job will clean the environment.

Scaling Factor The AS3AP benchmark scales by factor of 10.

Scalable Hardware Benchmark

The Scalable Hardware benchmark measures relational database systems. This benchmark is a subset of the AS3AP benchmark and tests the following:
l l l l

CPU Disk Network Any combination of the above three entities

How the Scalable Hardware Benchmark Works

The scale factor determines the amount of information initially loaded into the benchmark tables. For the Scalable Hardware benchmark, each scale factor represents one user accessing the system. Two tables are created in the database, and they are loaded with a varying number of rows. For each virtual user, a separate set of data must be created. Therefore the scale factor used when loading the database should be the size of the maximum user load. For example, with user loads of 1, 5, and 10, a scale factor of 10 should be used.

Scaling Factor
The Scalable Hardware benchmark has a scaling factor of one.

Best Practices
Load-testing against production servers Do not load-test against a production server if possible. Load-testing and benchmarking on a production server significantly degrades performance. In some cases, load-testing can cause a server to fail. However, if testing against a production server, take the following precautions:
l

Benchmark Factory for Databases User Guide Benchmarks

225

Determine how long it will take to restore a production server if it went down during load-testing Perform manual testing. Manual testing ensures that no unexpected outside activity takes place during the testing process

Reinitialize the Database To reinitialize a testing database, run a job containing a Benchmark Object node. There are two ways to create a Benchmark Object node:
l

Run the Benchmark Object Wizard to add the Delete Benchmark objects for the Scalable Hardware node to a job as needed. Run the Load Scenario Wizard to create a new script containing the Create Objects for the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the Benchmark. Running the delete objects for 'Scalable Hardware" job will clean the environment.

TPC-B Benchmark
l l l l

Overview Certification of Transaction Processing Council (TPC) Testing Results Best Practices Scaling Factor

Overview
The Transaction Processing Council is an organization that establishes transaction processing and database benchmark standards. For a complete overview and detailed explanation of the TPC-B Benchmark, please refer to: TPC-B Benchmark

Certification of Transaction Processing Council (TPC) Testing Results

Transaction Processing Council testing results cannot be published as certified unless the testing procedure is audited and approved by the TPC organization. If not certified, the testing results can be published as a "TPC-B like" test.

Best Practices
The following provides best practices for the TPC-B Benchmark.

Benchmark Factory for Databases User Guide Benchmarks

226

Load-testing against production servers Do not load-test against a production server if possible. Load-testing and benchmarking on a production server significantly degrades performance. In some cases, load-testing can cause a server to fail. However, if testing against a production server, take the following precautions:
l

l l

Reinitialize the Database To reinitialize a testing database, run a job containing a Benchmark Object node. There are two ways to create a Benchmark Object node:
l

Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-B node to a job as needed. Run the Load Scenario Wizard to create a new script containing the Create Objects for the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the Benchmark. Running the delete objects for 'TPC-B' job will clean the environment.

History Tables The TPC-B benchmark is made up of only one transaction that updates three tables and inserts a record into a history table. Inserting one record into one history table limits testing performance. The Benchmark Factory properties page allows the user to set the number of history tables to create during a test. The best ratio of history tables to virtual users is based on database configuration and hardware. The number of history tables to use is determined by the tester.

Scaling Factor
The TPC-B benchmark scales by a factor of one.

TPC-C Benchmark
l l

Overview TPC-C Tables

Benchmark Factory for Databases User Guide Benchmarks

227

l l

Certification of Transaction Processing Council (TPC) Testing Results Best Practices

Overview
Note: For a detailed overview of the TPC-C Benchmark, please refer to the Transaction Processing Council - TPC-C Benchmark. The TPC-C benchmark is an online transaction processing benchmark that simulates environments that have a number of terminal operators that send transactions to a database. This benchmark is focused on the concept of an order-entry type environment with transaction that include orders, payment recording, order status, and stock level monitoring. This benchmark portrays the activities of a wholesale supplier. However, the TPC-C is not limited to one particular business segment. It can represent numerous categories of a business that sell or distribute products and services. The TPC-C benchmark simulates a wholesale parts dealer operating out of warehouses. This Benchmark scales as a company, in theory, expands their business or number of facilities. As the TPC-C benchmark scales, so do the number components for the benchmark, for example, the sales districts and customers.

TPC-C Tables
The scale factor determines the amount of information initially loaded into the benchmark tables. For the TPC-C benchmark, each scale factor represents one warehouse as per TPC-C specification. The TPC-C benchmark involves a mix of five concurrent transactions of different types and complexity. The database is comprised of nine tables with a wide range of records. A maximum of 10 users should be run against each warehouse. For example, user loads of 1, 5, and 10, set the scale to 1. If using other user load values, change the scale factor accordingly. The TPC-C database consists of the following tables:

l l l l l l l

Warehouse District Customer History New_Order Order Order_Line

Benchmark Factory for Databases User Guide Benchmarks

228

l l

Item Stock

Certification of Transaction Processing Council (TPC) Testing Results

Best Practices
The following provides best practices for the TPC-C Benchmark. Load-testing against production servers Do not load-test against a production server if possible. Load-testing and benchmarking on a production server significantly degrades performance. In some cases, load-testing can cause a server to fail. However, if testing against a production server, take the following precautions:
l

l l

Reinitialize the Database To reinitialize a testing database, run a job containing a Benchmark Object node. There are two ways to create a Benchmark Object node:
l

Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-C node to a job as needed. Run the Load Scenario Wizard to create a new script containing the Create Objects for the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the Benchmark. Running the delete objects for 'TPC-C' job will clean the environment.

Benchmark Factory for Databases User Guide Benchmarks

229

TPC-D Benchmark
l l l l l

Overview TPC-D Transaction Details Certification of Transaction Processing Council (TPC) Testing Results Best Practices Scaling Factor

Certification of Transaction Processing Council (TPC) Testing Results

Best Practices
The following provides best practices for the TPC-D Benchmark. Load-testing against production servers Do not load-test against a production server if possible. Load-testing and benchmarking on a production server significantly degrades performance. In some cases, load-testing can cause a server to fail. However, if testing against a production server, take the following precautions:
l

l l

Reinitialize the Database To reinitialize a testing database, run a job containing a Benchmark Object node.

Benchmark Factory for Databases User Guide Benchmarks

230

There are two ways to create a Benchmark Object node:

Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-D node to a job as needed. Run the Load Scenario Wizard to create a new script containing the Create Objects for the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the Benchmark. Running the delete objects for 'TPC-D' job will clean the environment.

Scaling Factor
The TPC-D benchmark scales by the following factors:
l l l l l l

0.10 1.00 10.00 30.00 100.00 300.00

TPC-E Benchmark
l l l l

Overview Certification of Transaction Processing Council (TPC) Testing Results Best Practices Scaling Factor

Certification of Transaction Processing Council (TPC) Testing Results

Benchmark Factory for Databases User Guide Benchmarks

231

Best Practices
The following provides best practices for the TPC-E Benchmark. Load-testing against production servers Do not load-test against a production server if possible. Load-testing and benchmarking on a production server significantly degrades performance. In some cases, load-testing can cause a server to fail. However, if testing against a production server, take the following precautions:
l

l l

Scaling Factor
The TPC-E benchmark scales by factor of 500.

TPC-H Benchmark
l l l l

Overview Certification of Transaction Processing Council (TPC) Testing Results Best Practices Scaling Factor

Benchmark Factory for Databases User Guide Benchmarks

232

Certification of Transaction Processing Council Testing Results

Best Practices
The following provides best practices for the TPC-H Benchmark. Load-testing against production servers Do not load-test against a production server if possible. Load-testing and benchmarking on a production server significantly degrades performance. In some cases, load-testing can cause a server to fail. However, if testing against a production server, take the following precautions:
l

l l

Reinitialize the Database To reinitialize a testing database, run a job containing a Benchmark Object node. There are two ways to create a Benchmark Object node:
l

Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-H node to a job as needed. Run the Load Scenario Wizard to create a new script containing the Create Objects for the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the Benchmark. Running the delete objects for 'TPC-H' job will clean the environment.

Scaling Factor
The TPC-H benchmark scales by the following factors:
l l l l

1.00 10.00 30.00 100.00

Benchmark Factory for Databases User Guide Benchmarks

233

l l l l l l

300.00 1000.00 3000.00 10000.00 30000.00 100000.00

12
BFScripts
About Scripts
Note: All edit field with a yellow background are scriptable. Benchmark Factory provides a scripting capability as a means of inserting built-in functions for use in scripts. A function is a formula that takes one or more values (arguments), performs an operation, and returns a value that simulates real-world user activity. Functions are used alone or as building blocks in creating complex user activity. Randomized data is important when attempting to simulate real-world user activity because data that is random, prevents a server from using data stored in its cache. Click here for procedures on how to use scripts. The following provides a list of available scripts: Category Dates and Numbers Script Function $BFCreditCardExp $BFCurrentDate $BFCurrentDateTime $BFDate File Access Global Variables $BFFileArray $BFGetGlobalVar (page 240) $BFSetGlobalVar (page 240) $BFSetGlobalVarRtn (page 241) Name and Address $BFAddress $BFAddress2 $BFCity $BFCompany $BFCountry $BFEMail $BFFirstName $BFFullName $BFLastName $BFMiddleInitial $BFPhone $BFState $BFZipCode Numerical Manipulation $BFFormat

Benchmark Factory for Databases User Guide BFScripts

235

$BFProd $BFSum Random Numbers $BFRand $BFRandRange $BFURand (page 249) $BFURandRange Random Strings $BFCreditCard $BFRandList $BFRandMultiList $BFRandStr $BFURandList (page 253) $BFList String Manipulation $BFAsc $BFChr $BFConcat $BFLeft $BFLen $BFLower $BFMid (page 257) $BFRight $BFTrim $BFTrimLeft $BFTrimRight $BFUpper Test Info $BFGetVar $BFMaxNode $BFNode $BFNumberOfIterations $BFProfile $BFRunID $BFSetVar $BFSetVarRtn (page 263) $BFUserCounter $BFUserID $BFUserLoad $BFNextUserload $BFPrevUserload

Benchmark Factory for Databases User Guide BFScripts

236

Using Scripts in Username and Password Fields

Note: All edit field with a yellow background are scriptable. Scripts can be used in profiles to login with a variety of users and passwords. If scripts are to be used in a password field, the password needs to be contained entirely in a script such as $BFConcat. Example: $BFConcat("USER",$BFUserID(),"X") Will generate: "USER1x" for Virtual
User 1; "USER2x" for Virtual User 2; ...

This example can be useful to log in each virtual user with a unique username and password designated for this use. In this case users should be created in the test system with a username and password containing the virtual user number that will be used.

Dates and Numbers

$BFCreditCardExp
Description: Gets the global variable.

Parameters:

N/A

Syntax:

$BFGetGlobalVar('myvar')

$BFCurrentDate
Description:

Allows you to change the date format used to populate date fields.

Parameters:

Format string- The mix of static text and variables as needed.

Variables
%a %A

Definition Abbreviated weekday name

Full weekday name Abbreviated month name

%b
%B %c %d

Full month name

Date and time representing appropriate locale Day of month as decimal number (01-31)

Benchmark Factory for Databases User Guide BFScripts

237

%H
%I %j %m %M %p %S

Hour in 24-hour format (00-23) Hour in 12 hour format (01-12) Day of year as decimal number (001-366) Month as decimal number (01-12) Minute as decimal number (00-59) Current locale's A.M./P.M indicator for 12 hour clock Second as decimal number (00-59) Week of year as decimal number, with Sunday as first day of the week (00-53) Weekday as decimal number (0-6; Sunday is 0) Week of year as decimal number, with Monday as first day of the week (00-53) Date representation for current locale Time representation for current locale Year without century, as decimal (00-99) Year with century, as decimal number

%U
%w

%W
%x

%X
%y %Y %z, %Z

Either the time-zone name or the time zone abbreviation, depending on registry settings; no characters if time zone is unknown
Percent sign

Syntax:

$BFCurrentDate (Format String)

Example:

$BFCurrentDate ("%m/%d/%Y) ; returns 03/28/2005

BFCurrentDateTime
Description:

Allows you to change the date/time format used to populate date fields.

Parameters:

Format string- The mix of static text and variables as needed.

Variables
%a %A

Definition Abbreviated weekday name

Full weekday name Abbreviated month name

%b
%B %c %d

Full month name

Date and time representing appropriate locale Day of month as decimal number (01-31) Hour in 24-hour format (00-23) Hour in 12 hour format (01-12) Day of year as decimal number (001-366) Month as decimal number (01-12)

%H
%I %j %m

Benchmark Factory for Databases User Guide BFScripts

238

%M %p %S

%U
%w

Minute as decimal number (00-59) Current locale's A.M./P.M indicator for 12 hour clock Second as decimal number (00-59) Week of year as decimal number, with Sunday as first day of the week (00-53) Weekday as decimal number (0-6; Sunday is 0) Week of year as decimal number, with Monday as first day of the week (00-53) Date representation for current locale Time representation for current locale Year without century, as decimal (00-99) Year with century, as decimal number

%W
%x

%X
%y %Y %z, %Z

Either the time-zone name or the time zone abbreviation, depending on registry settings; no characters if time zone is unknown
Percent sign

Syntax:

$BFCurrentDateTime (Format String)

Example:

$BFCurrentDateTime ("%m/%d/%Y %H:%M:%S") ; returns 03/28/2005 14:18:52

$BFDate
Description: Randomly generates a date between a start date and a end date.

Parameters:

nStart -The start date. mm/dd/yyyy nEnd-The end date. mm/dd/yyyy

Syntax:

$BFDate(nStart,nEnd)

Example:

$BFDate("09/14/2004","09/14/2005") ; returns "11/04/2004"

Benchmark Factory for Databases User Guide BFScripts

239

File Access
$BFFileArray
Description: Selects an item from a list. Returns a single item from a comma-delimited file. The item returned depends on the mode selected. The syntax of the statement is also slightly different for each mode. Each virtual user gets a different seed value to generate unique sequences. Each agent machine must have a file with the name and path that is specified in the script function. If $BFFileArray is to return strings, the items in the file must be in double-quotes File name-The name and path of the source file. Retrieval Mode: CURRENT:Select the item in the specified column of the current row. RANDOM:Select a random item from a given column. SELECTION:Select an item based on column and row. SEQUENTIAL:Select each item sequentially. UNIQUE:Select a non-repeating item from a given column. nColumn-Which column of values to select. If none is given, a value of 1 is assumed. nRow-Which row to select for the item. Note: This value is only needed for the SELECTION mode.

Parameters:

Syntax:

$BFFileArray(FileName, Retrieval Mode[,nColumn][, nRow])

Example:

If myfile.dat consists of: "George Washington", "Martha" "Franklin Roosevelt","Eleanor" "Gerald Ford","Betty" $BFFileArray( "c:\path\myfile.dat", SELECTION, 2, 3) Returns Betty. $BFFileArray( "c:\path\myfile.dat", RANDOM,2) Returns either Martha, Eleanor, or Betty. Will return Martha, Eleanor, or Betty the next time it is run. $BFFileArray( "c:\path\myfile.dat", UNIQUE,2) Returns either Martha, Eleanor, or Betty. Will return Martha or Betty the next time it is run if it returned Eleanor the first time. $BFFileArray( "c:\path\myfile.dat", SEQUENTIAL, 1) Returns George Washington then Franklin Roosevelt then Gerald Ford. $BFFileArray( "c:\path\myfile.dat", CURRENT, 2) Returns Betty if the previous script function returned Gerald Ford.

Benchmark Factory for Databases User Guide BFScripts

240

Global Variable
$BFGetGlobalVar

Description:

Generates the GetGlobal Variable.

Parameters:

N/A

Syntax:

$BFGetGlobalVar('myvar')

$BFSetGlobalVar

Description:

Generates the Set Global Variable

Parameters:

N/A

Syntax:

$BFSetGlobalVar('myvar', 'my value')

Benchmark Factory for Databases User Guide BFScripts

241

$BFSetGlobalVarRtn

Description:

Generates the Set Global Variable Return.

Parameters:

N/A

Syntax:

$BFSetGlobalVarRtn('myvar', 'my value')

Name and Address

$BFAddress
Description: Returns a randomly generated street address string.

Parameters: Syntax:

N/A
BFAddress()

Example:

$BFAddress() ; returns "904 Oak Lane"

$BFAddress2
Description: Returns a second randomly generated street address string containing an apartment number, suite number, or villa. N/A

Parameters:

Benchmark Factory for Databases User Guide BFScripts

242

Syntax:

$BFAddress2()

Example:

$BFAddress2() ; returns

"Apt 5442"

$BFCity
Description: Generates a random city name.

Parameters:

N/A

Syntax:

$BFCity()

Example:

$BFCity() ; returns "Trend Blue of Asia"

$BFCompany
Description: Returns a random company name

Parameters:

N/A

Syntax:

$BFCompany()

Example:

$BFCompany(); returns "Quest Software, Inc."

Benchmark Factory for Databases User Guide BFScripts

243

$BFCountry
Description: Returns a randomly generated country string.

Parameters:

N/A

Syntax:

$BFCountry()

Example:

$BFCounty() ; returns "Canada"

$BFEmail
Description: Returns a random email address string.

Parameters:

N/A

Syntax:

$BFEmail()

Example:

$BFEmail() ; returns "Rita_Smyth@TrendBlue.net"

$BFFirstName
Description:

Returns a random first name string. N/A

$BFFirstName()

Parameters:

Syntax:

Example:

$BFFirstName() ; returns: "Hary D. Jones"

Benchmark Factory for Databases User Guide BFScripts

244

$BFFullName
Description:

Returns a random full name string.

Parameters:

N/A

Syntax:

$BFFullName()

Example:

$BFFullName() ; returns "Harry Jones"

$BFLastName
Description:

Returns a random last name string.

Parameters:

N/A

Syntax:

$BFLastName()

Example:

$BFLastName() ; returns "Jones"

$BFMiddleInitial
Description:

Returns a random middle initial character.

Parameters:

N/A

Syntax:

$BFMiddleInitial()

Benchmark Factory for Databases User Guide BFScripts

245

Example:

$BFMiddleInitial() ; returns "H"

$BFPhone
Description:

Returns a randomly generated telephone string.

Parameters:

N/A

Syntax:

$BFPhone()

Example:

$BFPhone() ; returns

"(348)889-6599"

$BFState
Description:

Returns a randomly generated state string.

Parameters:

N/A

Syntax:

$BFState()

Example:

$BFState() ; returns "CA"

Benchmark Factory for Databases User Guide BFScripts

246

$BFZipCode
Description:

Returns a randomly generated zip code string.

Parameters:

N/A

Syntax:

$BFZipCode()

Example:

$BFZipCode() ; returns "52076"

Numerical Manipulation
$BFFormat
Description:

Formats a series of up to 16 numbers. If the amount of numbers in the series is greater than 16, the Maximum Parameters Exceeded message displays. Format string -A %d for each number in the series. n1-The first number in the series. n2-The second number in the series. n16-The last number in the series.

Parameters:

Syntax:

$BFFormat(Format string, n1, n2, , n16)

Example:

$BFFormat("%d is a crazy number", $BFRandRange(10,200)) ; returns "34 is a crazy number"

Benchmark Factory for Databases User Guide BFScripts

247

$BFProd
Description:

Returns the product of a series of floating-point numbers.

Parameters:

f1-A floating-point number. f2- A second floating-point number. fn-The last floating-point number.

Syntax:

$BFProd(f1, f2, fn)

Example:

$BFProd(5, 2.5, .01) returns ; "0.125"

$BFSum
Description:

Returns the summation of a series of numbers.

Parameters:

f1-The first number to be summed. f2-The second number to be summed. fN-The last number to be summed.

Syntax:

$BFSum(f1, f2, , fN)

Example:

$BFSum(10, 20, 30) ; returns "60"

Benchmark Factory for Databases User Guide BFScripts

248

Random Numbers
$BFRand
Description:

Returns a random integer between 0 and nMax. Each virtual user gets a different seed value to generate the same unique sequences for each run. nMax-The maximum integer to be returned by the function.

Parameters:

Syntax:

$BFRand(nMax)

Example:

$BFRand(100) ; returns "45"

$BFRandRange
Description:

Returns a random integer ranging between nMin and nMax inclusive. Each virtual user gets a different seed value to generate sequences. nMin-The minimum integer in a range. nMax-The maximum integer in a range.

Parameters:

Syntax:

$BFRandRange(nMin,nMax)

Example:

$BFRandRange(200, 210) ; returns "202"

Benchmark Factory for Databases User Guide BFScripts

249

$BFURandRange
Description:

Returns unique integers ranging between the value of nMin and the value of nMax inclusive. Each virtual user gets a different seed value to generate unique sequences for each run. nMin-The minimum range integer to return. nMax-The maximum range integer to return.

Parameters:

Syntax:

$BFURandRange(nMin,nMax)

Example:

$BFRandRange(1,100) ; returns 100 the first time this function executes and 95, and 85 for subsequent executions of this function.

$BFURand
Description:

Returns a unique (non-repeating) random integer ranging between 1 and nMax. Each virtual user gets a different seed value to generate unique sequences for each run. nMax-The maximum integer to be returned.

Parameters:

Syntax:

$BFURand(nMax)

Example:

$BFURand(100) ;
returns 78 the first time this function executes, and 50, and 19 for subsequent executions of this function.

Benchmark Factory for Databases User Guide BFScripts

250

Random String
$BFCreditCard
Description:
Returns a random credit card number string.

Parameters:

N/A

Syntax:

$BFCreditCard()

Example:

$BFCreditCard() ; returns "7970 2384 8889 6599"

$BFRandList
Description:

Returns a string randomly selected from the list of items. If no weight is specified, a weight of 1 is assumed. Each virtual user gets a different seed value to generate unique sequences. string1-The first string in a list to return. nWeight1-Positive integer indicating the relative weight of the first string. string2-The second string in a list to return. nWeight2-Positive integer indicating the relative weight of the second string. stringN-The last string in a list to return.

Parameters:

Benchmark Factory for Databases User Guide BFScripts

251

nWeightN-Positive integer indicating the relative weight of the last string.

Syntax: $BFRandList(string1[:nWeight1], string2[:nWeight2], ,stringN[:nWeightN])

Example:

$BFRandList("Red", "White":2, "Blue":5) The value White will be returned 2 times more often than the value Red. The value Blue will be returned 5 times more often than the value Red.

$BFRandMultiList
Description:

Randomly selects multiple strings based on probabilities from a list. If Weight is omitted, a value of 100 is assumed. The probability that any string is include in the returned value is determined by the value of nWeight. Each string included in the return value is separated by a comma. string1-The first string in a list to return. nWeight1-Positive integer indicating the relative weight of the first string. Probability should be expressed as a value between 0 and 100. string2-The second string in a list to return. nWeight2-Positive integer indicating the relative weight of the second string. Probability should be expressed as a value between 0 and 100. stringN-The last string in a list to return. nWeightN-Positive integer indicating the relative weight of the last string. Probability should be expressed as a value between 0 and 100.

Parameters:

Syntax:

$BFRandMultiList(string1[:nWeight1], string2[:nWeight 2], , stringN"[: nWeightN])

Benchmark Factory for Databases User Guide BFScripts

252

Example:

$BFRandMultiList("Red", "White":50, "Blue":25) ; returns a value that always contains Red, contained White 50% of the transactions, and Blue 25% of the transactions.

$BFRandStr
Description: Returns a random string determined by a mode and having a length n. Length n-The length of a string. Mode constant-One of the following: ALPHA-Returns only random letters. ALPHA_LOWER-Returns random lowercase letters. ALPHA_NUM -Returns random letters and numbers. ALPHA_UPPER-Returns random uppercase letters. ANY-Returns any random ASCII characters. NUM -Returns only numbers. RANGE-Returns ASCII characters in the range specified. Syntax: $BFRandStr(n,[constant],[1..255],[1..255])

Parameters:

Example:

$BFRandStr(10) ; Returns 10 random characters. $BFRandStr(1,RANGE,65,67) ; Returns either A, B, or C.

Benchmark Factory for Databases User Guide BFScripts

253

$BFURandList
Description:

Randomly select unique items from a list.

Parameters:

string1-The first string to return from a list. string2-The second string to return from a list. stringN-The last string to return from a list.

Syntax:

$BFURandList(string1,string2, stringN)

Example:

$BFURandList("Red", "White", "Blue", "Yellow") ; returns Blue the first time this function executes then Yellow, Red, White, and Blue for subsequent executions of this function. Notice that Blue did not repeat until all other colors returned.

Benchmark Factory for Databases User Guide BFScripts

254

$BFList
Description: Returns an item from a list. The item returned depends on the mode selected. Retrieval Mode: RANDOM: Select a random item from the list. SEQUENTIAL: Select each item sequentially. UNIQUE: Select a non-repeating item from the list. string1-The first string to return from a list. string2-The second string to return from a list. stringN-The last string to return from a list. Syntax: $BFList(Retrieval Mode, string1,string2, stringN)

Parameters:

Example:

$BFList (Sequential "1", "2", "3", "4") Returns 1 2 etc. $BFList (Random 1, 2, 3, 4) Returns 2 3 3 1 2 4 etc. $BFList (Unique, "1", "2", "3","4") Returns 2 4 3 1 2 etc.

Benchmark Factory for Databases User Guide BFScripts

255

String Manipulation
$BFAsc
Description: Returns the ANSI value of the first character of a string.

Parameters:

String-Characters enclosed in quotation marks.

Syntax:

$BFAsc(string)

Example:

$BFAsc("George") ; Returns "71"

$BFChr
Description: Returns the character associated with the specified ANSI code. n-An integer representing an ANSI code.

Parameters:

Syntax:

$BFChr(n)

Example:

$BFChr(68) ; returns "D"

$BFConcat
Description: Returns a string containing two or more strings.

Parameters:

string1-The first string to return. string2-The second string to return. stringN-The last string to return.

Benchmark Factory for Databases User Guide BFScripts

256

Syntax:

$BFConcat("string1", "string2", , stringN)

Example:

$BFConcat ("The connection interval ", "determines how often an ", "agent attempts to connect to the Benchmark Factory Console.") ; returns "The connection interval determines how often an agent attempts to connect to the Benchmark Factory Console"

$BFLeft
Description:

Returns the first n character of a string.

Parameters:

String-Characters enclosed in quotation marks. nLength-The length of the string to return.

Syntax:

$BFLeft(string,length)

Example:

$BFLeft("Benchmark Factory", 5) ; returns

"Bench"

$BFLen
Description:

Returns the number of characters in a string.

Parameters:

String-Characters enclosed in quotation marks.

Syntax:

$BFLen(string)

Benchmark Factory for Databases User Guide BFScripts

257

Example:

$BFLen("Benchmark Factory") ; returns "17"

$BFLower
Description:

Returns a string after converting uppercase characters to lowercase characters. String-Characters enclosed in quotation marks.

Parameters:

Syntax:

$BFLowerstring)

Example:

$BFLower("SAMPLING") ; returns "sampling"

$BFMid
Description:

Extracts a substring from a string.

Parameters:

String-Characters enclosed in a quotation marks. nStart-The starting character position. nLength-Number of characters to return.

Syntax:

$BFMid(string, nStart,nLength)

Example:

$BFMid("sunshine coast", 3, 5) ; returns "shine"

Benchmark Factory for Databases User Guide BFScripts

258

$BFRight
Description:

Returns the last n character of a string.

Parameters:

String-Characters enclosed in quotation marks. nLength-The number of characters to return.

Syntax:

$BFRight(string,nLength)

Example:

$BFRight("sunshine coast", 3) ; returns "ast"

$BFTrim
Description: Returns a string void of leading and trailing spaces.

Parameters:

String-Characters enclosed in quotation marks.

Syntax:

$BFTrim(string)

Example:

$BFTrim(" happy days are here to stay. ") ; returns "happy days are here to stay"

$BFTrimLeft
Description:

Returns a string void of leading spaces.

Parameters:

String-Characters enclosed in quotation marks.

Benchmark Factory for Databases User Guide BFScripts

259

Syntax:

$BFTrimLeft(string)

Example:

$BFTrimLeft(" hockey is great ") ; returns " hockey is great "

$BFTrimRight
Description:

Removes trailing spaces from a string.

Parameters:

nString-Characters enclosed in quotation marks.

Syntax:

$BFTrimRight(string)

Example:

$BFTrimRight(" removes trailing spaces from a string. ") ; returns " removes trailing spaces from string. "

$BFUpper
Description:

Returns a string after converting lowercase characters to uppercase characters. nString value-Characters enclosed in quotation marks.

Parameters:

Syntax:

$BFUpper(string)

Example:

$BFUpper("sampling") ; returns "SAMPLING"

Benchmark Factory for Databases User Guide BFScripts

260

Test Info
$BFGetVar
Description:

Retrieves a previously stored value using $BFSetVar. Allows a value to be passed from one transaction to another in conjunction with $BFSetVar, or when value is used multiple times within a transaction. Each virtual user has its own variable space, so values are not shared between them. VarName-An alphanumeric identifier of the value stored.

Parameters:

Syntax:

$BFGetVar("VarName")

Example:

$BFSetVar("Totalrow", "2") $BFGetVar("Totalrow") ; if $BFSetVar sets the variable "Totalrow" to 2, $BFGetVar("Totalrow") returns the number 2.

$BFMaxNode
Description: Returns the total number of nodes for all users. This function is intended only for Oracle clustering. N/A

Parameters:

Syntax:

$BFMaxNode()

Example:

$BFMaxNode() ; returns: 2. This is the same value seen in the Number of Nodes field in Clustering tab for an Oracle profile.

Benchmark Factory for Databases User Guide BFScripts

261

$BFNode
Description:

Returns the node number of the current user. This function is intended only for Oracle clustering. N/A

Parameters:

Syntax:

$BFNode()

Example:

$BFNode() ; returns "1"

$BFNumberOfIterations
Description:

Returns the current number of iterations of a test.

Parameters:

N/A

Syntax:

$BFNumberOfIterations()

Example:

$BFNumberOfIterations() ; returns "1"

Benchmark Factory for Databases User Guide BFScripts

262

$BFProfile
Description:

Returns driver specific information, such as database name. Profile (constant)-The following provides a list of database type constants: MSSQL (native), SYBASE:
l l l

Parameters:

SERVERNAME DATABASE USERNAME

ORACLE:
l l

NETSERVICENAME USERNAME

ODBC, MYSQL (ODBC), MSSQL (ODBC):

l l

DATASOURCENAME USERNAME

DB2:
l l

DATABASEALIAS USERNAME

Syntax:

$BFProfile(constant)

Example:

$BFProfile(USERNAME) ; returns 'sa'

$BFRunID
Description:

Returns the run ID of the current test.

Parameters:

N/A

Benchmark Factory for Databases User Guide BFScripts

263

Syntax:

$BFRunID()

Example:

$BFRunID() ; returns "186"

$BFSetVar
Description:

Stores a value for later use by $BFGetVar. Used to store a value to be reused within its own transaction, or any transaction in a given user scenario. Each virtual user gets its own variable space, so values are not shared between them. Typically, $BFSetVar is placed at the beginning of a dynamic statement, as scripts are evaluated from left to right. Variable Name (VarName)-An alphanumeric identifier of the value stored. Text to Store (Value)-A string. The value to be stored for later retrieval.

Parameters:

Syntax:

Syntax:$BFSetVar("VarName", "Value")

Example:

$BFSetVar("Totalrow", "2") ; $BFSetVar sets the variable "Totalrow" to 2

$BFSetVarRtn
Description:

Stores and returns a value to be reused within its own transaction, or any transaction in a given user scenario. Each virtual user gets its own variable space, so values are not shared between them. Typically, $BFSetVarRtn is placed at the beginning of a dynamic statement, as scripts are evaluated from left to right. Variable Name (VarKey)-A string to store the a value.

Parameters:

Benchmark Factory for Databases User Guide BFScripts

264

Text to Store (Value)-A string value to be stored for later retrieval.

Syntax:

$BFSetVarRtn("Varkey","Value")

Example:

$BFSetVarRtn("Totalrows", "2") ; returns "2"

$BFUserCounter
Description:

Returns the user counter. N/A

Parameters:

Syntax:

$BFUserCounter()

Example:

$BFUserCounter() ; returns "1"

$BFUserID
Description: Returns the current virtual user ID.

Parameters:

N/A

Syntax:

$BFUserID()

Example:

$BFUserID() ; returns "1"

Benchmark Factory for Databases User Guide BFScripts

265

$BFUserLoad
Description:

Returns the current user load for the test running.

Parameters:

N/A

Syntax:

$BFUserload()

Example:

$BFUserLoad() ; returns "4"

$BFNextUserload
Description:

Returns the next user load for the load scenario running.

Parameters:

N/A

Syntax:

$BFNextUserload()

Example:

$BFNextUserload() When running with userloads 1, 4, 6, 10 this will return "6" when running at userload 4.

$BFPrevUserload
Description:

Returns the previous user load for the load scenario running. N/A

Parameters:

Benchmark Factory for Databases User Guide BFScripts

266

Syntax:

$BFPrevUserload()

Example:

$BFPrevUserload() When running with userloads 1, 4, 6, 10 this will return "4" when running at userload 6.

13
Quick Starts
About the Quickstarts
These quick start procedures introduce you to the Benchmark Factory environment. The intent of these procedures is not to show every aspect of the load testing process or every feature of Benchmark Factory. They provide the necessary background to quickly explore and begin using Benchmark Factory. The following quick starts are provided:
l l l l l

Real Application Cluster (RAC) Upgrading Your Database Server Using Benchmark Factory Oracle Trace Import-Database Replay Standard Benchmark Goal Testing

Real Application Clustering (RAC) Quick Start

Benchmark Factory for Databases User Guide Quick Starts

268

the cluster fail. Two important criteria used to achieve these goals focus on the cluster being able to load balance across all nodes, and when one or more nodes in the cluster fails, the users failover. When a node in an Oracle RAC cluster fails, an Oracle database and the remaining nodes continue to run, providing uninterrupted service to users. Oracle RAC distributes database demands across several nodes, performing the function of load balancing. When more capacity is required, additional nodes can be added with minimal effort. Organizations spending time and money setting up a RAC cluster must ensure their systems have the stability to withstand user environment demands when adding nodes.

Purpose of this Quick Start

Actions Performed During This Quick Start

l l l l l

Setting up the tnsnames.ora file Creating the Benchmark Factory cluster profile Running the RAC Load Test Importing an Oracle Trace File Viewing RAC Testing Results

Benchmark Factory for Databases User Guide Quick Starts

269

Figure 1-tnsnames.ora file set up for a three node cluster.

Benchmark Factory for Databases User Guide Quick Starts

270

Creating the Benchmark Factory Clustering Profile The profile contains information required to connect to the system-under-test, such as server name, IP address, and user name. To create the Benchmark Factory Clustering profile
1. Start Benchmark Factory if required . To start Benchmark Factory a. Click the Start button from the Windows task bar. b. Select Programs Quest Software Benchmark Factory for Database Benchmark Factory for Databases 5.5. Benchmark Factory launches. The Run Quick Start Dialog displays. Launch the Quick Start if desired. 2. Select the Script tab in the Script View. 3. Click Wizards | Load Scenario. 4. Click Next . The Load Scenario dialog displays. 5. Click Validate Oracle RAC Cluster Scalability. 6. Click Next . The Profile Selection dialog displays. 7. Click New. The Profile Creation Wizard displays. 8. Click Next . The driver selection dialog displays. 9. From the driver drop-down, select Oracle. 10. Click Next . The Connection Information dialog displays. 11. Fill in the required parameters.

Benchmark Factory for Databases User Guide Quick Starts

271

18. Select the desired Benchmark Scale. The larger the scale selection, the larger the database that will when created. It is suggested that a scale of 1 be selected, unless you desire a larger database to test with.

Running the RAC Load Test Benchmark Factory allows you to replay production workload and SQL scripts in different

Benchmark Factory for Databases User Guide Quick Starts

272

database environments, and places enormous stress on a database. By replaying production activity in a test database, potential performance problems can be identified before occurring in production. A system typically breaks under extreme load. By identifying system capacity and performance bottlenecks before they occur, Benchmark Factory facilitates proactive testing, which in turn reduces downtime, development costs, and potential loss of revenue. Benchmark Factory allows you to import a variety of user loads that include:
l

l l

Benchmark Factory for Databases User Guide Quick Starts

273

Benchmark Factory for Databases User Guide Quick Starts

274

Benchmark Factory for Databases User Guide Quick Starts

275

2. Right-click. A drop-down displays. 3. Select Submit Job. The Submit Job Dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Displays the Profile setting dialog.

Actions

Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Benchmark Factory for Databases User Guide Quick Starts

276

Section Comments

Description Allows you to add a comment to the job.

Alerts

Allows you to send an email, pager, or net-send operator alerts. Displays the Job Counter dialog.

Job Counters

Stop job after first error 4. Fill in the required parameters.

Stops the job if an error is encountered.

Benchmark Factory for Databases User Guide Quick Starts

277

2. You can select a single test or multiple test to view testing results. In this example, multiple tests are selected to view the RAC testing results.

3. Click Show Test Results. Run Reports Displays.

4. To view RAC testing results for the nodes selected, click Cluster Performance Graph.

Benchmark Factory for Databases User Guide Quick Starts

278

5. To view other testing results, click the desired result from the User Load drop-down.

Upgrading Your Database Server Quick Start

Overview
Microsoft SQL Server 2005 entry into the market forces many IT professionals to look at the upgrading of their current SQL Server 2000 Databases. IT organizations must perform testing before considering any major upgrades. Determining the capacity of a Microsoft SQL Server 2000 platform and comparing that to the performance of SQL Server 2005 is essential. Such tests help determine when an older version of your database will fail to scale, or become unstable, enabling you to compare those shortcomings to the newer versions of the database software. Before upgrading, three important questions must be asked:
l

Are you better able to meet important business requirements such as higher availability, better scalability, or better performance? Can you extract greater value out of the system by running it on the newest technology? Are you able to maintain the old application more efficiently than before?

l l

Purpose of this Quick Start

This quick start presents a load testing methodology to compare your current database servers

Benchmark Factory for Databases User Guide Quick Starts

279

with newer versions to determine if upgrading helps you obtain better performance.

Actions Performed During this Quick Start

l l l l l l l

Creating a SQL Server 2000 Trace File using SQL Server 2000 SQL Profiler Importing the Trace File Into Benchmark Factory for Databases Setting User Load Running the Load Test Against the SQL Server 2000 Running the Load Test Against the SQL Server 2005 Monitoring CPU Usage During the Test Viewing Testing Results

Creating the SQL Server 2000 Trace Files Using SQL Profiler Trace files obtain data in which events are logged during the running of a database application. You can then use the trace table to play back the logged events. You may have a trace table readily available, or can use Microsoft SQL Profiler to create a trace table. For procedures on creating SQL Server 2000 trace files, please refer to:
l

Creating a SQL 2000 Trace Table Using the SQL Server Profiler

Importing the Trace Table Into Benchmark Factory for Databases Once you have created the trace table, the next step is to import that trace table into Benchmark Factory for Database for the creation of a load scenario.
1. Start Benchmark Factory if required . To Benchmark Factory a. Click the Start button from the Windows task bar. b. Select Programs Quest Software Benchmark Factory for Database Benchmark Factory for Databases 5.5. Benchmark Factory launches. The Run Quick Start Dialog displays. Launch the Quick Start if desired.

2. Select the Script tab in the Script View.

3. Select Wizards | Load Scenario. 4. Click Next. The Select load scenario type dialog displays. 5. Select Replay load from SQL Server trace table. 6. Click Next. The SQL Server Authentication dialog displays. 7. Select the desired host from the Select a host drop-down. 8. Select the desired connection method. 9. Enter the user name and password if required.

Benchmark Factory for Databases User Guide Quick Starts

280

10. Click Next. The SQL Server Trace Table dialog displays. 11. Select the desired the desired database from the List of SQL Server databases on the selected host. 12. Select the desired trace table from List of SQL Server tables on the above database. 13. Click Next. The selected trace table processes. 14. When the processing completes, click Next. The Enter Script Name dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 15. Select Create a new Benchmark Factory Script file. 16. Enter the desired name in Name or accept the default. 17. Click Next. The completion dialog display. Do not check the Do you want the job to be automatically submitted to Jobs Queue. 18. Click Finish. The job creates and displays in the Script View. Setting User Load After importing the trace file, the next step is to set the timing and user load for the test. 1. In the Script view, select the load scenario replay that contains the imported trace table. 2. Right-click. 3. Select Copy to Mix Test. The Mix Test creates.

Benchmark Factory for Databases User Guide Quick Starts

281

4. Select the originally created replay.

5. Right-click. 6. Select Remove. 7. Click Yes when asked, "Are you sure you want to delete the selected item(s)? 8. Select the newly created Load Scenario. 9. In the properties view, select the Timing Tab. For this example, a 2:00 sampling length is selected.

10. In the properties view, select the User Load tab. For this example, user loads of 1 to 3000 are selected, with intervals of 250 Users.

Benchmark Factory for Databases User Guide Quick Starts

282

Running the Load Test on the SQL Server2000 Next, you run the workload on the 2000 SQL Server. 1. In the Script View, select the job. 2. Select Wizards | Submit Job. The Submit Job dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Displays the Profile setting dialog.

Actions

Allows you to:

Benchmark Factory for Databases User Guide Quick Starts

283

Section

Description Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Comments

Allows you to add a comment to the job.

Alerts

Allows you to send an email, pager, or net-send operator alerts. Displays the Job Counter dialog.

Job Counters

Stop job after first error

Stops the job if an error is encountered.

Note: You must select the 2000 SQL Server profile. 3. Fill in the required parameters. 4. Click Submit. The job runs and displays in the Jobs Queue. Running the Load Test on the SQL Server 2005 Next, you run the trace file on the 2005 SQL Server. 1. In the Script View, select the same job you used to run against the SQL 2000 server. (Imported trace file.)

Benchmark Factory for Databases User Guide Quick Starts

284

2. Select Wizards Submit Job. The Submit Job dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Displays the Profile setting dialog.

Actions

Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Comments

Allows you to add a comment to the job.

Benchmark Factory for Databases User Guide Quick Starts

285

Section

Description

Alerts

Allows you to send an email, pager, or net-send operator alerts. Displays the Job Counter dialog.

Job Counters

Stop job after first error

Stops the job if an error is encountered.

Note: You must select the 2005 SQL Server profile. 3. Fill in the required parameters. 4. Click Submit. The job runs and displays in the Jobs Queue. Monitoring Server Throughput During the Test As you run the test on the SQL Server, you can monitor the database using Spotlight on SQL Sever, to discover any issues that affect performance. Viewing Testing Results After running the test, you can use Benchmark Factory Run Reports to view testing results to see how SQL Server 2000 compared to SQL Server 2005. The transaction per second comparison between the two servers can be viewed. Figure 3 compares Transaction/Second vs. User Load graph of the 2000/2005 test. Run 13 (blue) represents the SQL Server 2000 and Run 16 (green) represents the SQL Server 2005. Note that in this example, SQL Server 2005 dramatically outperformed SQL Server 2000.

Benchmark Factory for Databases User Guide Quick Starts

286

Figure 3-Comparing SQL Server 2000 to SQL Server 2005 Transaction/Second The following graphic compares the Transaction Time between SQL Server 2000 and SQL Server 2005. Note that in this example, the SQL Server 2005 dramatically outperformed SQL Server 2000.

Oracle Trace Import-Database Replay Quick Start

Purpose of this Quick Start
This quick start presents a load testing methodology to import and run an Oracle trace file against the AS3AP benchmark, then view testing results. Using trace files in a database load test allows you to run recorded production activity on a test database that includes concurrency between sessions, session activity, and latencies that include bind variables.

Benchmark Factory for Databases User Guide Quick Starts

287

Actions Performed During This Quick Start

l l l l l l l

Accessing the Benchmark Factory Script and Oracle trace files Creating the AS3AP tables Creating a profile Importing the Oracle trace files Submitting the Job to the Jobs Queue Viewing run reports Saving the script

Accessing the Benchmark Factory script and Oracle Trace Files for this Quick Start Industry standard benchmarks have been developed to measure system performance. Each benchmark has specific system configuration requirements that include tables, indexes, and data. For this Quick Start, we will use the AS3AP Benchmark to create the environment to run the test on. The Oracle Trace File Setup.bfp, OraDemo_2.trc, and OraDemo_2.trc files are provided to create the AS3AP tables and run the load test. The following files are located in C:\Program Files\Quest Software\Benchmark Factory for Databases\Quick Starts:
l l l

Oracle Trace File Setup.bfp OraDemo_1.trc OraDemo_2.trc

Creating the AS3AP Tables The Oracle Trace File Setup.bfp has been provided to create the tables needed to run the job created by the Oracle trace files.
1. Start Benchmark Factory if required . To start Benchmark Factory a. Click the Start button from the Windows task bar. b. Select Programs Quest Software Benchmark Factory for Database Benchmark Factory for Databases 5.5. Benchmark Factory launches. The Run Quick Start Dialog displays. Launch the Quick Start if desired.

2. Select the Script tab in the Script View. 3. Click File | Open. 4. Navigate to C:\Program Files\Quest Software\Benchmark Factory for Databases\Quick Starts. 5. Select Oracle Trace File Setup.bfp.

Benchmark Factory for Databases User Guide Quick Starts

288

6. Click Open. The Oracle Trace File Setup opens and displays in the script view. 7. Next, you must create the AS3AP tables to run the job. Select the AS3AP Tables job.

8. Select the AS3AP Tables job. 9. Select Wizards | Submit Job. The Submit Job dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Displays the Profile setting dialog.

Benchmark Factory for Databases User Guide Quick Starts

289

Actions

Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Comments Alerts Job Counters Stop job after first error

10. Fill in the desired parameters.

Allows you to add a comment to the job. Allows you to send an email, pager, or net-send operator alerts. Displays the Job Counter dialog. Stops the job if an error is encountered.

11. Click Submit . The load test is submitted to the Jobs Queue and runs.

Importing the Oracle Trace Files After creating the AS3AP tables, you create the Oracle trace file script. 1. Click the script tab located at the bottom of the Jobs Queue. 2. Select Wizards | Load Scenario. 3. The Load Scenario Wizard displays. The Load Scenario Wizard is a starting point for creating Benchmark Factory scripts and walks you through the steps to build a complete workload testing script. Click Next. 4. Click Replay load from an Oracle Trace file. 5. Click Next. The Oracle 8i/9i/10g/11g trace file(s) to import window displays. 6. Click on the Add Trace button. The Select File Location window displays. 7. Navigate to the Benchmark Factory Quick Start directory. (C:\Program Files\Quest Software\Benchmark Factory for Databases\Quick Starts.) 8. While holding down the Ctrl key, click on the OraDemo_1.trc and OraDemo_2.trc. 9. Click Open. The trace files display under the Trace File Name column. 10. Click the Next button. The Oracle Trace Activity window displays. 11. Click the Import entire trace file(s) radio button. 12. Click Next. Select the Create a New Benchmark Factory Script file radio button. 13. Enter the desired name, such as Oracle Trace File Import. 14. Click Next. The wizard completion dialog displays.

Benchmark Factory for Databases User Guide Quick Starts

290

15. Uncheck the "Do you want the job to be automatically submitted to the jobs Queue?" check box. 16. Click Finish. The Submit Job Wizard displays.

Submitting the Job to the Jobs Queue The job can now be submitted to the jobs queue. 1. Select the Jobs node. 2. Right-click and select Submit Job. The Submit Job dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Benchmark Factory for Databases User Guide Quick Starts

291

Section

Description Displays the Profile Creation Wizard.

Displays the Profile setting dialog.

Actions

Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Comments

Allows you to add a comment to the job.

Alerts

Allows you to send an email, pager, or net-send operator alerts. Displays the Job Counter dialog.

Job Counters

Stop job after first error

Stops the job if an error is encountered.

3. Fill in the required parameters. 4. Click Submit. The load test is submitted to the Jobs Queue and runs. Note: For user loads of 20 users or less, the Agent is not required.

Benchmark Factory for Databases User Guide Quick Starts

292

Viewing Run Reports Results 1. Click the Run Reports node for the Oracle Trace Import job.

2. Select the desired run.

3. Double-click. Run Reports opens to the Run ID node. This view contains a summary of job statistics. 4. Expand Summary Graphs node. Summary graphs provide a wide array of graphs that allow you to view in detail your testing results. 5. Expand the Results node. Results provides information on individual user load and agent load testing activity. 6. Close Run Reports when finished reviewing the job. Saving the Script 1. Click the script tab in the Script View. 2. Select File | Save. The Save As dialog displays. 3. Enter the desired script name in File Name. 4. Click Save.

Benchmark Factory for Databases User Guide Quick Starts

293

Standard Benchmark Quick Start

Purpose of this Quick Start
This quick start presents a load testing methodology to run an AS3AP benchmark on a test database, then view real-time and stored testing results to determine if your database lives up to the required performance parameters. A benchmark is a performance test of hardware or software. A benchmark simulates real-world application workloads. These models of real applications can then be run on the system being evaluated. Models works better than the actual application and offers reproducible results. The real application, on the other hand, has too many variables that can change the results over several test sessions. Benchmarks measure performance that analyze:
l l l l l l

Raw performance of a complete system Create a profile Raw performance of a specific subsystem (disk, video, CPU, memory, etc.) Performance of a computer running a particular application Performance of applications running on a network Capacity of a system. This is often referred to as capacity planning.

Actions Performed During this Quick Start

l l l l l l l

Creating a database load test script Create a profile Setting the benchmark scale and user load Submit a Job to the Job Queue View real-time results View run reports Save the Script

Benchmark Factory for Databases User Guide Quick Starts

294

Creating a Database Load Test Script

1. Start Benchmark Factory if required . To start Benchmark Factory a. Click the Start button from the Windows task bar. b. Select Programs Quest Software Benchmark Factory for Database Benchmark Factory for Databases 5.5. Benchmark Factory launches. The Run Quick Start Dialog displays. Launch the Quick Start if desired.

2. Select the Script tab in the Script View.

3. Select Wizards|Load Scenario. The Load Scenario Wizard displays. The Load Scenario Wizard is the starting point for all Benchmark Factory scripts. The New Script Wizard walks you through the steps to build a complete load testing script. 4. Click Next. The Select Load Scenario dialog displays. 5. Click Industry standard benchmarks (TPC-C, TPC-H...). Benchmark Factory provides industry Standard Benchmarks that allows you to populate test databases according to a selected industry standard benchmark and measures the performance of a system-under-test. Benchmark Factory provides the following standard benchmarks:
l l l l l l

AS3AP Scalable Hardware TPC-B TPC-C TPC-D TPC-H

6. Click Next. The Select Benchmark dialog displays. This dialog displays the standard benchmarks that come with Benchmark Factory. For a complete overview of all Benchmarks, please refer to Benchmark Overview. 7. Click the AS3AP benchmark.

Benchmark Factory for Databases User Guide Quick Starts

295

8. Click Next. The Profile Selection dialog displays. Creating a Profile 1. If no profile has been created, click New. The Profile Creation Wizard displays. If a profile has been created, select the desired profile from the Profile drop-down. 2. Click Next. The Benchmark Scale dialog displays. Setting the Benchmark Scale and User Load 1. Benchmark Factory allows you to set the desired benchmark scale to perform realistic load tests. Select the desired benchmark scale in Benchmark Scale. 2. Click Next. The Measurement Intervals dialog displays. 3. Add the desired User Load. 4. Click Next. The Enter Script Name dialog displays and provides the following options:
l

5. Select the Create a new Benchmark Factory script radio button. 6. Enter the desired name or accept the default. 7. Click Next. The wizard completion dialog displays. 8. Check the Do you want the job to be automatically submitted to the jobs Queue? check box.

Benchmark Factory for Databases User Guide Quick Starts

296

9. Click Finish. The Submit Job dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Displays the Profile setting dialog. Actions Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit. Comments Allows you to add a comment to the job.

Benchmark Factory for Databases User Guide Quick Starts

297

Alerts Job Counters Stop job after first error 10. Fill in the required parameters.

Allows you to send an email, pager, or net-send operator alerts. Displays the Job Counter dialog. Stops the job if an error is encountered.

11. Click Submit. The job runs and displays in the Jobs Queue. Viewing Realtime Results During the running of a database load test, real-time statistics provide insight into the performance of the database being tested. Numerous datapoints allow you to examine exactly how your database is performing. Note: For procedures on changing Benchmark Factory graph views, please refer to Changing Graph Views. 1. Click the Message node on the Jobs Queue tab. The Message node displays Job status, error messages, and general information.

2. Expand the Run Status node. Run Status provides a real-time status on the test currently running in the Jobs Queue. 3. Click the Realtime Statistics node. The Real Time Statistics provides you with real-time graphs and raw data. This data allows you to spot system-under-test issues that may be affecting server performance. The Real Time Statics node displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows you to change graph settings and view. 4. Click Click here to Add Datapoints. 5. Select the desired datapoints. 6. Click OK. 7. Right-click on the graph. 8. Select Graph Legend.

Benchmark Factory for Databases User Guide Quick Starts

298

9. Click the Summary node. The Summary Graph displays. The Summary page displays user selected statistics for the job timing period following completion of an iteration. 10. From the Vs. Userload drop-down in the upper left-hand corner of the graph, select the desired datapoint.

11. Click the Transaction Graph node. The transaction node displays properties based on transaction type. 12. From the Vs. Userload drop-down in the upper left-hand corner of the graph, select the desired datapoint.

Benchmark Factory for Databases User Guide Quick Starts

299

Viewing Run Reports Results Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark Factory and provides a comprehensive and detailed collection of load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. 1. Click the Run Reports node.

2. Select the desired run.

Benchmark Factory for Databases User Guide Quick Starts

300

3. Double-click. Run Reports opens to the Run ID node. This view contains an summary of load scenario statistics. 4. Expand Summary Graphs node. Summary graphs provide a wide array of graphs that allow you to view in detail your testing results. 5. Expand the Results node. Results provides information on individual user load and agent load testing activity. 6. Close Run Reports when finished reviewing the load scenario. Saving the Script After running a job, you can save a job to run again, or edit to run using different testing parameters. 1. Click the Script tab in the Script View. 2. Select File | Save. The Save As dialog displays. 3. Enter the desired script name in File Name. 4. Click Save.

Goal Testing Quick Start

Purpose of this Quick Start

Benchmark Factory for Databases User Guide Quick Starts

301

A Goal Test uses a transaction mix A transaction is a single unit of work in the testing process, used in load scenariosand user scenarios. For example, a SQL statement. based on userload intervals. The userloads are determined by setting a beginning, ending value, and interval value. This value specifies an interval to increase the number of users tested for each iteration of the Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration, and the test ends once the goal or maxim user load has been reached.

Actions Performed During This Quick Start

l l l l l l l l

2. Select the Script tab in the Script View. 3. Select Wizards | Load Scenario. The Load Scenario Wizard displays. The Load Scenario Wizard is the starting point for all Benchmark Factory scripts. The Load Scenario Wizard walks you through the steps to build a complete load testing script. 4. Click Next. The Select Load Scenario dialog displays. 5. Click the Create a custom load scenario (Scalability, Goal...). 6. Click Next. The Test Type dialog displays. 7. Click the Goal radio button. 8. Click Next. The Goal Page displays. The Goal page allows you to:

Benchmark Factory for Databases User Guide Quick Starts

302

10. Select the desired option. For this quick start, the Find maximum TPS is selected. 11. Set the desired Start, Ending, and Interval in Userloads. For this test, the defaults values of 1, 5, and 1 are used. When run, User Loads of 1, 2, 3, 4, 5 are run until the TPS of one run is less than the TPS of the previous run. 12. Click Next. The Define Transaction Mix Page displays. 13. Click Next. The Enter Script Name dialog displays and provides the following options: 2. Create a new Benchmark Factory Script file-Selecting this option allows you to create and save a completely new script. 3. Add to existing Benchmark Factory Script file-Selecting this option allows you to add the load scenario you are building to the current script file you are working with. 14. Click Create a new Benchmark Factory Script file. 15. Click Next. The wizard completion dialog displays. 16. Click Finish. The Scripts creates and displays in the Script View.

Benchmark Factory for Databases User Guide Quick Starts

303

Setting Script Timing 1. Click the Max TPS (1-5 by 1) node.

5. Click the Start all users as quickly as possible radio button. Adding a Transaction to the Goal Test 1. Click the Max TPS (1-5 by 1) node.

Benchmark Factory for Databases User Guide Quick Starts

304

2. Select Wizards | Transaction. The Transaction Wizard displays.

7. Click Next. The Real World Latencies dialog displays. 8. Select Keying Time. 9. From the Keying Time drop-down select Normal.

Benchmark Factory for Databases User Guide Quick Starts

305

10. In Mean enter 3000 milliseconds.

11. Click Next. The wizard completion dialog displays. 12. Click Finish. The transaction creates and displays in the Script View. Creating Benchmark Objects 1. In order to create the a_updates table used in the SQL statement, Benchmark Objects must be created. Click the Max TPS (1-5 by 1) job node. 2. Right-click. A drop-down displays. 3. Select Add Create/Delete Benchmark Objects. The Benchmark Objects Wizard displays.

Benchmark Factory for Databases User Guide Quick Starts

306

13. Move the Benchmark Object to the top node in the script view.

Benchmark Factory for Databases User Guide Quick Starts

307

Submitting the job to run 1. Click the Max TPS(1-5 by 1) job node.

2. Select Wizards | Submit Job Wizard. The Submit Job dialog displays.

The Submit Job dialog allows you to tailor your job needs and provides the following options:

Section Profile

Description Allows you to select profiles already created.

Displays the Profile Creation Wizard.

Benchmark Factory for Databases User Guide Quick Starts

308

Section

Description

Displays the Profile setting dialog.

Actions

Allows you to: Run Job: Runs the job. No action: Closes the dialog after you select Submit.

Comments

Allows you to add a comment to the job.

Alerts

Allows you to send an email, pager, or net-send operator alerts.

Job Counters

Displays the Job Counter dialog.

Stop job after first error

Stops the job if an error is encountered.

Benchmark Factory for Databases User Guide Quick Starts

309

4. Click the Summary node. The Summary Graph displays. The Summary page displays user selected statistics for the job timing period following completion of an iteration. 5. Click the Transaction Graph node. The Summary Graph displays. The transaction node displays properties based on transaction type. Viewing Run Reports Results 1. When the job completes, click the Run Reports node.

14
Troubleshooting
Agent Connection
If the agent is having a problem connecting to the Benchmark Factory console, please check the following
l l

Verify that the console is running. Verify that the agent is pointed to the console machine. Go to Options > Settings and check the Machine Name/IP Setting. Verify TCP connectivity between the Agent and Console. If there is a firewall in place, allow for communication on Port #4568.

Use Benchmark Factory with SQL Server 2005 Client

Microsoft SQL Server 2005 does not include the DB-Library used by the Benchmark Factory "Microsoft SQL Server (Native)" driver. Either the installed SQL Native ODBC driver or another Microsoft SQL Server ODBC driver can be used with the Benchmark Factory "Microsoft SQL Server (ODBC)" driver to connect to a "Microsoft SQL Server" database. If you prefer to use the more optimized "Microsoft SQL Server (Native)" driver, a pre-SQL Server 2005 client can be installed to provide the DB-Library.

MySQL Initialization Settings

The MySQL repository database can take up a large amount of space on a system. The my.ini file allows you to edit the initialization settings and adjust the size of the MySQL repository used by Benchmark Factory. For additional information on setting MySQL database parameters, please visit MySQL.com.

15
Appendix
Change Graph Views
Right-clicking on a Benchmark Factory graph displays a drop-down that allows you to customize graph settings. Whether you are viewing a real-time graph or one in Benchmark Factory Run Reports, Benchmark Factory uses the same graphing tool. In most cases you will find that the graphs are presented in the most meaningful form (Line Graphs or Bar Graphs based on the data being presented.) Displays Benchmark Factory Run Reports Shows in a data form or graph form Displays the Graph Legend Displays the graph toolbar Prints the graph Copies Data to Clipboard Copies Graph to Clipboard Loads a saved graph configuration Saves a graph configuration Sets the current configuration as default Clears the current chart configuration

Show Data/Show Graph

Toggling to Graph displays the data in graph form.

Benchmark Factory for Databases User Guide Appendix

312

Graph Legend
Toggling to Graph Legend displays a legend on the side of the graph.

Toolbar
Toggling to Toolbar displays the graph toolbar.

Benchmark Factory for Databases User Guide Appendix

313

Print
Choosing Print displays the Print Dialog.

Copy Data to Clipboard

Copies the data to the clipboard.

Copy Graph to Clipboard

Copies the graph to the clipboard.

Load Configuration
Benchmark factories graphs allows you to save graph configurations.

Save Configuration
Saves a graph configuration.

Set as Default
Sets a configured graph as default.

Clear Configuration
Clears a graph configuration.

Customize List Controls

Benchmark Factory allows you to customize list controls in columns to select the desired viewing format. To Customize Lists Controls 1. Right-click the desired column. A menu displays.

Benchmark Factory for Databases User Guide Appendix

314

2. The following options are provided:

l l l l

Sort Ascending: Sorts columns in ascending order. Sort Descending: Sorts columns in descending order. Customize Columns: Allows you to customize column headings. Column Format: Allows you to customize column format.

Create a SQL 7 Trace Table Using the SQL Server Profiler

To create a SQL 7 trace table 1. Initiate the SQL Profiler. 2. Click the New Trace icon. The Connect to SQL Server window displays. 3. Login to the SQL server. The Trace Properties window displays. 4. Type a name of a trace file into Trace Name. 5. Check the Capture To Table box. The Capture TO Table window displays. 6. Select a database name from the Database drop-down list. 7. Select dbo from the Owner drop-down list. 8. Click OK. 9. Click on the Events tab. 10. Select TSQL from the Available events classes list. 11. Click the Add button. 12. Click the Data Columns tab. 13. Add the following columns (minimal): 14. EventClass 15. SQLUserName 16. ConnectionID 17. NTUserName 18. Duration 19. StartTime 20. Text 21. DatabaseID 14. Click OK. 15. Run SQL statements. 16. When finished running SQL statements, click the Stop this trace icon. The table creates.

Benchmark Factory for Databases User Guide Appendix

315

Create a SQL 2000 Trace Table Using the SQL Server Profiler
To create a SQL 2000 trace table 1. Initiate the SQL Profiler. 2. Click the New Trace icon. The Connect to SQL Server window displays. 3. Select the appropriate SQL server from the drop-down list. 4. Check the Start SQL Server if it is stopped check box. 5. Select the appropriate Connect using radio button. Type the appropriate login name and password if appropriate. 6. Click OK. The Trace Properties window displays. 7. Click the General tab. 8. Type a name of the trace into Trace name. 9. Check the Save to table check box. The Destination Table window displays. 10. Select the appropriate database from the Database drop-down list. 11. Select the appropriate table from the Table drop-down list. 12. Click OK. The Trace Properties window displays. 13. Click the Events tab. 14. Select TSQL from the Available events classes list. 15. Click Add. 16. Click the Data Columns tab. 17. Add the following columns (minimal): 15. EventClass 16. Login Name 17. ClientProcessID 18. NTUserName 19. Duration 20. StartTime 21. TextData 18. Click Run. 19. Run SQL statements.

Benchmark Factory for Databases User Guide Appendix

316

20. When finished running SQL statements, click the Stop selected trace icon. The table creates.

Create a SQL 2005 Trace Table Using the SQL Server Profiler
To create a SQL 2005 trace table 1. Initiate the SQL Profiler. 2. Click the New Trace icon. The Connect to SQL Server window displays. 3. Select the appropriate server name from the drop-down list. 4. Select the appropriate authentication from the drop-down. Type the login name and password if required. 5. Click Connect. The Trace Properties window displays. 6. Type a name of the trace into Trace name. 7. Check the Save to table check box. The Connect to Server dialog displays. 8. Select the appropriate server name from the drop-down list. 9. Select the appropriate authentication from the drop-down. Type the login name and password if required. 10. Click Connect. The Destination Table window displays. 11. Select the appropriate database from the Database drop-down list. 12. Select the appropriate owner from Own drop-down. 13. Select the appropriate table from the Table drop-down list. 14. Click OK. The Trace Properties window displays. 15. Click the Events Selection tab. 16. Verify that all check boxes are checked in the TSQL-SQL:BatchStarting Events, and TSQL-SQL:BatchComplete Events. 17. Click Run. 18. Run SQL statements. 19. When finished running SQL statements, click the Stop selected trace icon. The table creates.

Benchmark Factory for Databases User Guide Appendix

317

Oracle Instant Client Installation

To run Benchmark Factory, you must have the client dll files installed. To install the Oracle 10g Instant Client dlls 1. Place Oracle 10g instant Client on your system following the directions provided by Oracle. For these instructions, please visit the Oracle 10g Web site. 2. Copy tnsnames.ora from a configured system to your system. 3. Edit your systems Environment Variables using one of the following methods: 4. Add to the path variable the directory the Instant Client files were copied to. 5. Add a new System variable Named TNS_Admin, and place the directory location of the tnsnames.ora file in the value field. Note: Environment Variables can be reached by right clicking on the My Computer Icon, selecting properties, select Advanced tab, and select the Environmental Variables button.

Migrating Repository Data Using the DOS Command Line

The Repository is a database where all of the test results are stored. Benchmark Factory inserts test results into the repository and provides an easy way to access the data. By default, the Repository is a MySQL database that resides on the same machine as Benchmark Factory for Databases. The Repository can reside on another database server if required. To change the database, select the Repository Manager or the repository tab in Settings. Benchmark Factory allows you to migrate data from the DOS command line. To migrate data using the DOS command line 1. Click Start | All Programs | Accessories | Command Prompt. 2. Navigate to the Benchmark Factory for Databases bin directory. Example: CD C:\Program Files\Quest Software\Benchmark Factory for Databases\bin. 3. To display the syntax help, type:
bmfdatamigrationWizard.exe -?.

4. Press Enter. The usage parameters display.

Benchmark Factory for Databases User Guide Appendix

318

5. To perform the migration use the following syntax:

BMFDataMigrationWizard -s [DSN,USER,PWD] -d [DSN,USER,PWD]

Example:
BMFDataMigrationWizard -s [MyDatabase,root,yourpassword] d[LocalServer,sa,sa]

Store Procedure Examples

The following provides syntax examples of statements that call stored procedure. Click here for procedures on creating a load scenario by importing stored procedures. Database Oracle Examples Example with parameters: BEGIN YOUR_PROC(:VAR1,:VAR2); END; Bind Parameter Settings Parameter :VAR1 :VAR2 Type INTEGER INTEGER Value 5 8

Example without parameters: BEGIN YOUR_PROC2 ( );

Benchmark Factory for Databases User Guide Appendix

319

END;

Microsoft SQL Server and Sybase

Example with parameters: exec YOUR_PROC(@VAR1,@VAR2) Bind Parameter Settings Parameter @VAR1 @VAR2 Type INTEGER INTEGER Value 55 77

Example without parameters: exec YOUR_PROC2 IBM DB2, ODBC, and MySQL Example with parameters: call procedure-name (?,?,?,?); Bind Parameter Settings Parameter 1 2 3 4 Type STRING INTEGER INTEGER INTEGER Value A 5 7 0

Example without parameters: call procedure-name ();

Benchmark Factory for Databases User Guide Appendix

320

Note: When using an ODBC connection, the syntax for the statement will follow the syntax for the database, with the exception of the parameters. The parameters within the statement would be "?" and the bind parameters names should be 1,2,3,4...

Oracle Trace File Activation

Benchmark Factory allows you to import an Oracle trace files for use during database testing. The following provides Oracle commands that can be used to enable Oracle trace file activity. Note: Please consult Oracle documentation for current trace file commands. To turn on and off the trace use the ALTER SESSION command: 1. Set the TIMED_STATISTICS and MAX_DUMP_ FILE_SIZE parameters used by the sessions:
alter session set timed_statistics=true alter session set max_dump_file_size=unlimited

2. Activate the trace:

alter session set events '10046 trace name context forever, level 4'

3. Stop the trace:

alter session set events '10046 trace name context off'

To activate a trace you don't have read/write access to: 1. Acquire the V$SESSION.SID and V$SESSION.SERIAL# values of the session you wish to trace.

Benchmark Factory for Databases User Guide Appendix

321

2. Set the TIMED_STATISTICS and MAX_DUMP_FILE_SIZE parameters for the session utilizing the following procedure calls:
dbms_system.set_bool_param_in_session( sid => 55, serial# => 1222, parnam => 'timed_statistics', bval => true) dbms_system.set_int_param_in_session( sid => 55, serial# => 1222, parnam => 'max_dump_file_size', intval => 2147483647)

Note: If using a release before Oracle 8, release 8.1.6, these parameters can be changed with ALTER SYSTEM commands. 3. Activate the trace.
dbms_support.start_trace_in_session( sid => 55, serial => 1222, waits => true, binds => true)

4. Stop the trace.

dbms_support.stop_trace_in_session( sid => 55, serial => 1222)

Support Bundle
The Benchmark Factory Support Bundle allows you to zip up a .bfp file and send it to support for review. The bundle contains the following information:

Benchmark Factory for Databases User Guide Appendix

322

l l l l l

BMF version number Settings License information (just send the key files) Error Logs and Result Logs from the associated directories Files located in the data directory. This will be not only the xml files for imported users scenarios, but also dump files. Script files XML configuration files A file that contains hardware information about the system running on it as well as the output of the agents configurations from the repository Information about versions of all loaded .dlls

l l

To create a Benchmark Factory support bundle: 1. Select Help| Support Bundle. 2. Select the required modules to send. 3. Click Bundle. 4. The bundle is created and saved in: C:\Documents and Settings\"User Name"\My Documents\My Benchmark Factory\BMFSupportBundle.zip.

System Requirements/Upgrade Requirements/Supported Databases

System Requirements
Before installing Benchmark for Databases , ensure your system meets the following minimum hardware and software requirements. Repository Benchmark Factory cannot run without a repository, and by default a MySQL repository is installed during the install process. After installation, you can remove the MySQL service and delete the MySQL repository, then select a repository of your choice. Benchmark Factory supports the following databases:

Benchmark Factory for Databases User Guide Appendix

323

l l l l

Oracle SQL Server MySQL Sybase Adaptive Server Anywhere (iAnywhere)

Memory 512 MB of RAM recommended Note: The memory required may vary based on the following:
l l l l

Applications that are running on your system. Size and complexity of the database. Amount of database activity. Number of concurrent users accessing the database.

Platform 1.0 GHz x86 or x64 compatible CPU Hard Disk Space 120 MB of disk space for installation Note: The disk space required may vary based on options selected for installation. Operating System Microsoft
l l l l l l

Windows 2000 Window 2003 Windows 2008 XP Professional Vista Windows 7

Database Server Oracle: 8, 8i, 9i, 9.2, 10g, and 11g (Requires Oracle 8.1.7 client libraries or higher.) Note: Oracle Capture/Replay of workloads must use Oracle 9i or higher. SQL Server: 7, 2000, 2005, and 2008 Sybase: 12.5 and 15

Benchmark Factory for Databases User Guide Appendix

324

DB2 LUW: 8.1.5 through 9 MySQL: 4.5 and 5.x Note: All client libraries require the 32-bit versions. Upgrade Requirements
l

Client libraries for database types used during the workload testing process must be installed on all testing machines (Benchmark Factory and Agents). The Benchmark Factory 3.3 and below files can be converted to Benchmark Factory for Databases 4.x files. A script conversion program is available, and can be accessed by logging on to Quest Software, then select Downloads and Upgrades | Benchmark Factory for Databases| Special Patches| Benchmark Factory Project Converter.zip. There is no upgrade path for the Benchmark Factory Repository version 3.3 or below. If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark Factory will not work against this repository.

l l

Shortcut Keys
The following provides a list of shortcut keys used in Benchmark Factory. Key ALT+1 ALT+2 ALT+M ALT+R ALT+S CTRL+B CTRL+C CRTL+D CTRL+E CTRL+F CTRL+J Action Displays the Output View Displays the Agent window Creates an email message with the current script attached Runs a job Stops a job Displays the Benchmark Objects Wizard Copies the selected item in the Script View Saves a job back to the Script View Readies a job Displays the Execute File Wizard (Script view only) Displays the Submit to Job Wizard (Script view only)

Benchmark Factory for Databases User Guide Appendix

325

Key

Action

CTRL+L CTRL+N CTRL+O CTRL+P CTRL+R CTRL+S CTRL+T CTRL+U CTRL+V CTRL+W CTRL+X CTRL+ ALT+E CTRL+H CTRL+ALT+H Delete END HOME LEFT ARROW RIGHT ARROW F1 F8 F12 PLUS SIGN on numeric keypad (+)

Displays the Load Scenario Wizard (Script view only) Displays the Load Scenario Wizard (Script view only) Displays the Select Script File to Open dialog (Script view only) Displays the Edit Profile dialog Displays the Enter a New Name dialog (Script view only) Saves the current script (Script view only) Displays the Transaction Wizard (Script view only) Displays the User Scenario Wizard (Script view only) Paste an item from the clipboard (Script view only) Displays the Profile Creation Wizard Cuts a selected item (Script view only) Readies all jobs Holds a job Holds all jobs Deletes the selected item (Script view only) Selects the end of the item with focus Selects the beginning of the item with focus Collapses current selection if it is expanded, or select parent folder Expands current selection if it's collapsed, or select first subfolder

Displays online help Displays the Setting dialog Displays the Save As dialog Expands the selected folder

Benchmark Factory for Databases User Guide Appendix

326

Key LOCK+MINUS SIGN on numeric keypad (-)

Action Collapses the selected folder

16
Copyright, Licensing, and Licensing Agreement
Copyright

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

328

Quest, Quest Software, the Quest Software logo, AccessManager, ActiveRoles, Aelita, Akonix, Benchmark Factory, Big Brother, BridgeAccess, BridgeAutoEscalate, BridgeSearch, BridgeTrak, BusinessInsight, ChangeAuditor, CI Discovery, Defender, DeployDirector, Desktop Authority, Directory Analyzer, Directory Troubleshooter, DS Analyzer, DS Expert, Foglight, GPOADmin, Help Desk Authority, Imceda, IntelliProfile, InTrust, Invirtus, iToken, JClass, JProbe, LeccoTech, LiteSpeed, LiveReorg, LogADmin, MessageStats, Monosphere, NBSpool, NetBase, NetControl, Npulse, NetPro, PassGo, PerformaSure, Point, Click, Done!, Quest vToolkit, Quest vWorkSpace, ReportADmin, RestoreADmin, ScriptLogic, SelfServiceADmin, SharePlex, Sitraka, SmartAlarm, Spotlight, SQL Navigator, SQL Watch, SQLab, Stat, StealthCollect, Storage Horizon, Tag and Follow, Toad, T.O.A.D., Toad World, vAutomator, vConverter, vEcoShell, VESI,vFoglight, vPackager, vRanger, vSpotlight, vStream, vToad, Vintela, Virtual DBA, VizionCore, Vizioncore vAutomation Suite, Vizioncore vEssentials, Vizioncore vWorkflow, WebDefender, Webthority, Xaffire, and XRT are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. For a complete list of Quest Softwares trademarks, please see http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered trademarks are property of their respective owners.

Benchmark Factory for Databases User Guide January 2011

Licensing and Virtual User Packs

The Benchmark Factory for Databases Licensing dialog allows you to modify the license and virtual user packs. To access the licensing dialog: 1. Click Help | Licensing.

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

329

From this dialog you can:

l l

Modify the Benchmark Factory License (page 329) Modify the Benchmark Factory Virtual User Pack (page 330)

Modify the Benchmark Factory License

If required, you can modify your Benchmark Factory license key. To to modify the license key: 1. Click the Benchmark Factory tab. 2. Click Modify. 3. From the License Type drop-down, select the required Quest product. The following licenses can be updated:
l l l l l

Toad for Oracle Benchmark Factory for Databases Quest Central for Oracle Load Generator key Quest Central for SQL Server Load Generator key Quest Central for DB2 Load Generator key

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

330

4. Click Edit Key. 5. Update the Authorization Key and Site Message. 6. Click OK. 7. Click Close.

Modify the Benchmark Factory Virtual User Pack

Benchmark Factory comes with 100 virtual users. Benchmark Factory Freeware comes with 20 virtual users. To add up to an unlimited number of virtual users, enter a virtual user key. To enter one of the following key types, select the desired key from the drop-down and click the Edit key. Click here to learn more about adding virtual users. To to modify the VirtualUser Pack: 1. Click the Virtual User Pack tab. a. To send an email to Quest Support to extend a trial, click Extend Trial. b. To visit the Quest web site and purchase a license, click Purchase License. c. To modify the currant license, click Modify. d. Click Virtual User Key. e. Enter the authorization key and site message. f. Click OK. g. Click OK. h. Click Close.

Transaction Product Agreement

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

331

PLEASE READ THIS AGREEMENT CAREFULLY BEFORE USING THIS PRODUCT. BY DOWNLOADING, INSTALLING OR USING THIS PRODUCT, YOU INDICATE ACCEPTANCE OF AND AGREE TO THE TERMS AND CONDITIONS OF THIS AGREEMENT. FOR ORDERS PLACED OUTSIDE OF THE UNITED STATES, PLEASE SEE WWW.QUEST.COM/LICENSEAGREEMENTS FOR THE APPLICABLE LOCALIZED VERSION OF YOUR AGREEMENT. IF YOU DO NOT AGREE TO THE TERMS AND CONDITIONS OF THIS AGREEMENT OR THE LOCALIZED AGREEMENT, DO NOT DOWNLOAD, INSTALL OR USE THIS PRODUCT. NEITHER THIS AGREEMENT NOR THE LOCALIZED AGREEMENT SHALL SUPERSEDE ANY OTHER SIGNED AGREEMENT BETWEEN YOU AND QUEST THAT EXPRESSLY GOVERNS THE ORDER FOR THIS PRODUCT. This Transaction Product Agreement (the Agreement) is made between Quest Software, Inc., with its principal place of business located at 5 Polaris Way, Aliso Viejo, California, 92656 (Quest) and you, the customer (Customer). 1. Definitions. Capitalized terms not defined in context shall have the meanings assigned to them below: (a) Affiliate means any legal entity controlling, controlled by, or under common control with a party to this Agreement, for so long as such control relationship exists. (b) Documentation means the user manuals and documentation that Quest delivers with the Software, and all copies of the foregoing. (c) Hardware means the hardware products purchased by Customer under this Agreement. (d) License Type means the model by which the Software is licensed (e.g., by server, by mailbox, by managed user) as may be indicated in the applicable Order. (e) Partner means a reseller or distributor that is under contract with Quest or another Partner and is authorized via the contract to resell the Products and/or Maintenance Services. (f) Product Guide means the document located at http://www.quest.com/productguide that contains the Product Terms. (g) Product Terms means the usage rights and other terms associated with each License Type or individual Product. The Product Terms for Software identified in an Order that is issued to Quest shall be as stated in the Order, or, if no Product Terms are stated in the Order, then the Product Terms for such Software shall be as stated in the Product Guide as of the date of the Order. The Product Terms for Software ordered from a Partner shall be as stated in the Product Guide as of the date of the Order. (h) Products means the Software licenses and/or Hardware purchased by Customer under this Agreement. (i) Order means the document by which Customer orders the Product(s) or, if Customer orders the Product(s) from Quests eStore (https://estore.quest.com), the process by which Customer orders the Product(s). Orders executed by Customer and Quest shall be governed solely by the terms of this Agreement and the applicable Order. Orders placed with Quest by Customer purchase order only and all Orders placed through a Partner shall be governed solely by the

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

332

terms of this Agreement. Any conflicting or additional terms in or accompanying an Order will not be binding on Quest unless Quest accepts such terms in writing. Each Order shall be the Customers irrevocable commitment to purchase and pay for the Products and/or Maintenance Services stated in the Order. (j) Software means the object code version of the software that is delivered pursuant to an Order as well as any corrections, enhancements, and upgrades to such software that Quest may provide to Customer pursuant to this Agreement, and all copies of the foregoing. 2. Software License. (a) Internal Use License. Subject to the terms of this Agreement, Quest grants to Customer, and Customer accepts from Quest, a perpetual (unless otherwise set forth in an Order), non-exclusive, non-transferable (except as otherwise set forth herein) and non-sublicensable license to (i) install, execute, access, run, or otherwise use the quantities of each item of Software identified in the applicable Order within the parameters of the Product Terms associated with the applicable Product and License Type, (ii) make a reasonable number of additional copies of the Software to be used solely for non-productive archival or passive disaster recovery purposes, so long as neither the original and a copy nor two copies of the same Software are used at the same time, and (iii) make and use copies of the Documentation as reasonably necessary to support Customers authorized users in their use of the Software (collectively, License). Except for MSP Licenses (as defined below), each License shall be used by Customer solely to manage its own internal business operations as well as the business operations of its Affiliates. (b) MSP License. If an MSP License is specifically identified in an Order, Customer shall be granted a License to use the Software identified in the Order and the associated Documentation as a managed service provider (MSP) to provide software and systems management services, including, without limitation, application, operating system, and database implementation, performance tuning, and maintenance services (Management Services), for the benefit of a single named client (Client), pursuant to the terms of this Agreement and the MSP Use Terms in the Product Guide. (c) Evaluation License. If an Order indicates that Software is to be used for evaluation purposes or if the Software is otherwise obtained from Quest for evaluation purposes, Customer shall be granted a non-production License to use such Software and the associated Documentation solely for Customers own internal evaluation purposes for an evaluation period of up to thirty (30) days from the date of delivery of the Software, plus any extensions granted by Quest in writing (the "Evaluation Period"). There is no fee for Customers use of the Software for non-production evaluation purposes during the Evaluation Period, however, Customer is responsible for any applicable shipping charges or taxes which may be incurred, and any fees which may be associated with usage beyond the scope permitted herein. Customers opportunity for a free evaluation of the Software is limited to one Evaluation Period per release of the Software. Notwithstanding anything otherwise set forth in this Agreement, Customer understands and agrees that evaluation Software is provided AS IS and that Quest does not provide a Warranty or Maintenance Services for evaluation Licenses. (d) Third Party Use. If Customer contracts with a third party who performs Software implementation, configuration, consulting or outsourcing services (Service Provider), the Service Provider may use the Software and Documentation Licensed by Customer hereunder

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

333

solely for purposes of providing such services to Customer, provided that (i) Customer ensures that the Service Provider uses the Software and Documentation in accordance with the terms of this Agreement, (ii) the use of the Software and Documentation by the Service Provider will not violate the terms of the export restrictions set forth herein, and (iii) the Service Provider is not a Quest competitor. Customer shall be jointly and severally liable to Quest for the acts and omissions of its Service Providers in connection with their permitted use of the Software and Documentation. (e) Freeware. If a freeware version of Quest software (Freeware) is downloaded by Customer from http:///a Quest website, the terms of such use shall be governed by the applicable Freeware definition provided at: www.quest.com/productguide. 3. Restrictions. Except to the extent expressly permitted by applicable law, and to the extent that Quest is not permitted by such applicable law to exclude or limit the following rights, Customer may not reverse engineer, decompile, disassemble, or attempt to discover or modify in any way the underlying source code of the Products, Documentation or any part thereof. In addition, Customer may not (i) modify, translate, localize, adapt, rent, lease, loan, create or prepare derivative works of, or create a patent based on the Products, Documentation or any part thereof, or (ii) resell the Products or Documentation or use the Products or Documentation in any commercial time share arrangement, in connection with the operation of any nuclear facilities, or for purposes which are competitive to Quest. Each permitted copy of the Software and Documentation made by Customer hereunder must contain all titles, trademarks, copyrights and restricted rights notices as in the original. Customer understands and agrees that the Products may work in conjunction with third party products and Customer agrees to be responsible for ensuring that it is properly licensed to use such third party products. Notwithstanding anything otherwise set forth in this Agreement, the terms and restrictions set forth herein shall not prevent or restrict Customer from exercising additional or different rights to any open source software that may be contained in or provided with the Products in accordance with the applicable open source licenses. 4. Reservation of Rights and Ownership. Quest reserves any and all rights, implied or otherwise, which are not expressly granted to Customer in this Agreement. Customer understands and agrees that (i) the Products are protected by copyright and other intellectual property laws and treaties, (ii) Quest and/or its suppliers own the title, copyright, and other intellectual property rights in the Products, (iii) the Software is licensed, and not sold, and (iv) this Agreement does not grant Customer any rights to Quests trademarks or service marks. 5. Hardware. In the event Customer acquires Hardware under this Agreement, title to such Hardware shall pass to Customer upon shipment (unless such Hardware is rented, leased or loaned to Customer). 6. Payment. Customer agrees to pay to Quest (or, if applicable, the Partner) the fees specified in each Order, including any applicable shipping fees. Customer will be invoiced promptly following delivery of the Products or prior to the commencement of any Renewal Maintenance Period and Customer shall make all payments due to Quest in full within thirty (30) days from the date of each invoice or such other period (if any) stated in an Order signed by Quest. Any amounts payable to Quest by Customer that remain unpaid after the due date shall be subject to a late charge of 1.5% of the invoice amount per month from the due date until such amount is paid, or the maximum rate permitted by law if less.

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

334

7. Taxes. The fees stated in an Order may not include taxes. If Quest is required to pay sales, use, property, value-added or other taxes based on the Products or Maintenance Services provided under this Agreement or on Customers use of Products or Maintenance Services, then such taxes shall be billed to and paid by Customer. This Section does not apply to taxes based on Quests income. 8. Termination. This Agreement and/or the License(s) granted hereunder may be terminated (i) by mutual agreement of Quest and Customer, (ii) by Quest, if Customer or a Service Provider commits a material breach of this Agreement and fails to cure such breach to Quests reasonable satisfaction within thirty (30) days following receipt of Quests notice thereof, or (iii) by Customer for any reason upon thirty (30) days written notice to Quest. Upon termination of this Agreement or expiration or termination of a License for any reason, all rights granted to Customer for the applicable License(s) shall immediately cease and Customer shall immediately: (i) cease using the applicable Software and Documentation, (ii) return the applicable Software to Quest together with all Documentation and other materials associated with the Software and all copies of any of the foregoing, or destroy such items, (iii) cease using the Maintenance Services associated with the applicable License(s), (iv) pay Quest or the applicable Partner all amounts due and payable up to the date of termination, and (v) give Quest a written certification that Customer has complied with all of the foregoing obligations. Termination of this Agreement or a License shall be without prejudice to any other remedies that the terminating party may have under law, subject to the limitations and exclusions set forth in this Agreement. Any provision of this Agreement that requires or contemplates execution after termination of this Agreement or expiration of a License is enforceable against the other party and their respective successors and assignees notwithstanding termination or expiration, including, without limitation, the Payment, Taxes, Termination, Warranty Disclaimer, Infringement, "Limitation of Liability," "Nondisclosure," Usage Verification, and General Sections of this Agreement. 9. Export. Customer acknowledges and agrees that the Products are subject to the export control laws, rules, regulations, restrictions and national security controls of the United States and other applicable foreign agencies (the "Export Controls"), and agrees not to export or re-export, or allow the export or re-export of the Products or any copy, portion or direct product of the foregoing in violation of the Export Controls. Customer hereby represents that (i) Customer is not an entity or person to which shipment of Products is prohibited by the Export Controls; and (ii) Customer will not export, re-export or otherwise transfer the Products to (a) any country subject to a United States trade embargo, (b) a national or resident of any country subject to a United States trade embargo, (c) any person or entity to which shipment of Products is prohibited by the Export Controls, or (d) anyone who is engaged in activities related to the design, development, production, or use of nuclear materials, nuclear facilities, nuclear weapons, missiles or chemical or biological weapons. 10. Maintenance. During any Maintenance Period and for the applicable fees, Quest shall make available to Customer the Maintenance Services for the Software as defined in this Section. The first Maintenance Period begins on the date of delivery of the Software following an Order and ends twelve (12) months thereafter unless otherwise set forth in the applicable Order (the Initial Maintenance Period). Following the Initial Maintenance Period, Maintenance Services shall automatically renew for additional terms of twelve (12) months (each, a Renewal Maintenance Period) unless the renewal has been cancelled by either party giving written notice to the other at least sixty (60) days prior to the first day of the applicable Renewal Maintenance Period.

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

335

Cancellation of Maintenance Services will not terminate Customers rights to continue to use the Software. Maintenance fees shall be due in advance of a Renewal Maintenance Period and shall be subject to the payment requirements set forth in this Agreement. The procedure for reinstating Maintenance Services after it has lapsed is posted at http://support.quest.com/Maintenance_ Service.asp. Except as otherwise stated in the Product Guide, Maintenance Services shall be available via the Internet, e-mail, or telephone and shall mean the following: (a) Quest shall make available to Customer new versions and releases of the Software, including Software corrections, enhancements and upgrades, if and when Quest makes them generally available without charge as part of Maintenance Services. (b) Quest shall respond to unlimited communications from Customer that report Software failures not previously reported to Quest by Customer. Nothing in the foregoing shall operate to limit or restrict follow up communication by Customer regarding Software failures. (c) Quest shall respond to requests from Customers technical coordinators for assistance with the operational/technical aspects of the Software; provided that Quest shall have the right to limit such responses if Quest determines, in its sole reasonable discretion, that on-site consulting services would be more appropriate to address the scope and nature of the requests. Any such onsite consultation would be pursuant to a services agreement as agreed upon by the parties. (d) Customer shall have access to Quests Support Web site at http://support.quest.com (SupportLink). (e) Maintenance Services are available during standard support hours (Business Hours) as indicated on SupportLink. In addition, Customer may purchase Business Critical Support (i.e. 24x7 Severity Level 1 support) for certain Software. The list of Software for which Business Critical Support is available and/or required is set forth on SupportLink. (f) During Business Hours, Quest will respond within one (1) hour to a call from Customer which reports a critical Software condition (a Severity Level 1 Problem). Customer must use commercially reasonable efforts to provide Quest with the necessary remote access to facilitate the identification and resolution of a Severity Level 1 Problem. Quests ability to identify and resolve a Severity Level 1 Problem may be delayed without such remote access. (g) The Maintenance Services for those Software products that Quest has obtained through an acquisition or merger may, for a period of time following the effective date of the acquisition or merger, be governed by terms other than those in this Section 10. The applicable different terms, if any, shall be stated on SupportLink. 11. Warranties. (a) Software Warranty. Quest warrants that, for a period of thirty (30) days following the initial delivery of Software pursuant to an Order (the Warranty Period), (i) the media provided by Quest, if any, on which the Software is recorded will be free from material defects in materials and workmanship under normal use, (ii) the operation of the Software, as provided by Quest, will substantially conform to the Documentation applicable to such Software, and (iii) the Software as delivered by Quest does not contain any viruses, worms, Trojan Horses, or other malicious or destructive code designed by Quest to allow unauthorized intrusion upon, disabling of, or erasure of the Software (however, the Software may contain a key limiting use of the Software to

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

336

within the scope of License granted, and license keys issued by Quest for temporary use are timesensitive) (the Warranties). Customer must give written notice to Quest of any breach of the Warranties no later than five days following the expiration of the Warranty Period. Customers exclusive remedies, and Quests sole obligations, for any such breach of these Warranties shall be as follows: (a) for the warranty in subsection (i), Quest shall, at its expense, replace any defective media; (b) for the warranty in subsection (ii), Quest shall correct or provide a workaround for reproducible errors in the Software that cause a breach of the warranty within a reasonable time considering the severity of the error and its effect on Customer, or, at Quests option, refund the license fees paid for the nonconforming Software upon return of such Software to Quest and termination of the related License(s) hereunder; and (c) for the warranty in subsection (iii), Quest shall provide a copy of the Software that is in conformance with such warranty. The foregoing Warranties shall not apply to any non-conformance (i) that Quest cannot recreate after exercising commercially reasonable efforts to attempt to do so; (ii) caused by misuse of the Software or by using the Software in a manner that is inconsistent with this Agreement or the Documentation; or (iii) arising from the modification of the Software by anyone other than Quest. (b) Hardware Warranty. Hardware shall be warranted in accordance with the warranty document delivered with the Hardware and/or included on the hardware manufacturers website. In the event Customer acquires Hardware that is delivered with a third party warranty (Third Party Warranty), Customer will rely solely on the applicable third party for all Third Party Warranty obligations. (c) Warranty Disclaimer. THE EXPRESS WARRANTIES AND REMEDIES SET FORTH IN THIS SECTION ARE THE ONLY WARRANTIES AND REMEDIES PROVIDED BY QUEST HEREUNDER. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, ALL OTHER WARRANTIES OR REMEDIES ARE EXCLUDED, WHETHER EXPRESS OR IMPLIED, ORAL OR WRITTEN, INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, NONINFRINGEMENT, SATISFACTORY QUALITY, AND ANY WARRANTIES ARISING FROM USAGE OF TRADE OR COURSE OF DEALING OR PERFORMANCE. QUEST DOES NOT WARRANT UNINTERRUPTED OR ERROR-FREE OPERATION OF THE PRODUCTS. 12. Infringement. Quest will at its own expense defend or settle any claim, suit, action, or proceeding brought against Customer by a third party to the extent it is based on an allegation that the Software directly infringes any patent, copyright, trademark, or other proprietary right enforceable in the country in which the Software is delivered to Customer, or misappropriates a trade secret in such country (a Claim). Additionally, Quest shall pay any judgments finally awarded against Customer under a Claim or any amounts assessed against Customer in any settlements of a Claim, and reasonable administrative costs or expenses, including without limitation reasonable attorneys fees, necessarily incurred by Customer in responding to the Claim. Quests obligations under this Section are conditioned upon Customer (i) giving prompt written notice of the Claim to Quest; (ii) permitting Quest to retain sole control of the investigation, defense or settlement of the Claim, and (iii) providing Quest with such cooperation and assistance as Quest may reasonably request from time to time in connection with the investigation, defense or settlement of the Claim. Quest shall have no obligation hereunder to

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

337

defend Customer against any Claim (a) resulting from use of the Software other than as authorized in this Agreement, (b) resulting from a modification of the Software other than by Quest, or (c) based on Customers use of the Software after Quest recommends discontinuation because of possible or actual infringement, (d) based on Customers use of a superseded or altered release of Software if the infringement would have been avoided by use of a current or unaltered release of the Software made available to Customer, or (e) to the extent the Claim arises from or is based on the use of the Software with other products, services, or data not supplied by Quest if the infringement would not have occurred but for such use. If Customers use of the Software is enjoined as a result of a Claim, Quest shall, at its expense and option either (i) obtain for Customer the right to continue using the Software, (ii) replace the Software with a functionally equivalent non-infringing product, (iii) modify the Software so that it is non-infringing, or (iv) accept the return of the infringing Software and refund the license fee paid for the infringing Software, pro-rated over a sixty (60) month period from the date of delivery of the Software following an Order . This Section states the entire liability of Quest, and Customers sole and exclusive remedy, with respect to a Claim. 13. Limitation of Liability. EXCEPT FOR (A) ANY BREACH OF THE "RESTRICTIONS" OR "NONDISCLOSURE" SECTIONS OF THIS AGREEMENT, (B) AMOUNTS CONTAINED IN JUDGMENTS OR SETTLEMENTS WHICH QUEST IS LIABLE TO PAY ON BEHALF OF CUSTOMER UNDER THE INFRINGEMENT SECTION OF THIS AGREEMENT, OR (C) ANY LIABILITY TO THE EXTENT LIABILITY MAY NOT BE EXCLUDED OR LIMITED AS A MATTER OF LAW, IN NO EVENT SHALL QUEST, ITS AFFILIATES, OR SUPPLIERS, OR CUSTOMER BE LIABLE FOR ANY LOSS OF REVENUE, LOSS OF ACTUAL OR ANTICIPATED PROFITS, LOSS OF BUSINESS, LOSS OF CONTRACTS, LOSS OF GOODWILL OR REPUTATION, LOSS OF ANTICIPATED SAVINGS, LOSS OF, DAMAGE TO OR CORRUPTION OF DATA, OR FOR ANY INDIRECT, INCIDENTAL, SPECIAL OR CONSEQUENTIAL LOSS OR DAMAGE OF ANY KIND, IN EACH CASE HOWSOEVER ARISING, WHETHER SUCH LOSS OR DAMAGE WAS FORESEEABLE OR IN THE CONTEMPLATION OF THE PARTIES AND WHETHER ARISING IN OR FOR BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), BREACH OF STATUTORY DUTY, OR OTHERWISE. EXCEPT FOR (A) ANY BREACH OF CUSTOMERS PAYMENT OBLIGATIONS; (B) ANY BREACH OF THE "SOFTWARE LICENSE," RESTRICTIONS," EXPORT OR "NONDISCLOSURE" SECTIONS OF THIS AGREEMENT, OR ANY OTHER VIOLATION OF THE OTHER PARTYS INTELLECTUAL PROPERTY RIGHTS; (C) QUESTS EXPRESS OBLIGATIONS UNDER THE INFRINGEMENT SECTION OF THIS AGREEMENT; OR (D) ANY LIABILITY TO THE EXTENT LIABILITY MAY NOT BE EXCLUDED OR LIMITED AS A MATTER OF LAW, THE MAXIMUM AGGREGATE AND CUMULATIVE LIABILITY OF QUEST, ITS AFFILIATES AND SUPPLIERS, AND CUSTOMER UNDER THIS AGREEMENT, WHETHER ARISING IN OR FOR BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), BREACH OF STATUTORY DUTY, OR OTHERWISE, SHALL NOT EXCEED THE FEES PAID AND/OR OWED (AS APPLICABLE) BY CUSTOMER FOR THE PRODUCTS OR MAINTENANCE SERVICES THAT ARE THE SUBJECT OF THE CLAIM. FOR MAINTENANCE SERVICES OR A PRODUCT SUBJECT TO RECURRING FEES, THE LIABILITY SHALL NOT EXCEED THE AMOUNT PAID AND/OR OWED (AS APPLICABLE) FOR SUCH MAINTENANCE SERVICE OR PRODUCT DURING THE TWELVE (12) MONTHS PRECEDING THE CLAIM.

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

338

Quests Affiliates and suppliers shall be beneficiaries of this Limitation of Liability" section and Customers Service Providers are entitled to the rights granted under the "Third Party Use" section of this Agreement; otherwise, no third party beneficiaries exist under this Agreement. Quest expressly excludes any and all liability to Customers Service Providers, Clients and to any other third party. 14. Nondisclosure. Confidential Information means information or materials disclosed by one party (the Disclosing Party) to the other party (the Receiving Party) that are not generally available to the public and which, due to their character and nature, a reasonable person under like circumstances would treat as confidential, including, without limitation, the Disclosing Partys personal data, financial information, marketing information, trade secrets, know-how, proprietary tools, proprietary knowledge and proprietary methodologies. Confidential Information of Quest further includes this Agreement, the Products (in source code and/or object code form), the pricing and discounting offered by Quest to Customer hereunder, information regarding the functionality and performance of the Products, benchmark test results regarding the Products, and any Software license keys provided to Customer. Additionally, Confidential Information shall include Individually Identifiable Health Information (as that term is defined in 45 CFR 164.501) or Nonpublic Personal Information (as that term is defined in Title V of the GrammLeach-Bliley Act of 1999) that Quest may come into contact with under this Agreement. Confidential Information shall not include information or materials that (a) were, on the date of disclosure, generally known to the public; (b) become generally known to the public other than as a result of the act or omission of the Receiving Party; (c) were known to the Receiving Party without an obligation of confidentiality prior to that party receiving the same from the Disclosing Party; (d) the Receiving Party lawfully received from a third party without that third partys breach of agreement or obligation of trust; or (e) are or were independently developed by the Receiving Party without access to or use of the Disclosing Partys Confidential Information. Additionally, it shall not be a breach of this Section for the Receiving Party to disclose the Disclosing Partys Confidential Information as may be required by operation of law or legal process, provided that the Receiving Party provides prior notice of such disclosure to the Disclosing Party unless expressly prohibited from doing so by a court, arbitration panel or other legal authority of competent jurisdiction. The Receiving Party shall not (1) make the Disclosing Partys Confidential Information available to any Affiliates, directors, officers, employees, consultants or representatives (collectively, the Representatives) who do not have a need to know in order to carry out the purposes of this Agreement; (2) otherwise disclose the Disclosing Partys Confidential Information to any third party without the written consent of the Disclosing Party; or (3) use the Disclosing Partys Confidential Information for any purpose other than as contemplated by this Agreement. The Receiving Party shall inform its Representatives of the confidential nature of the Disclosing Partys Confidential Information and the requirements regarding restrictions on disclosure and use as set forth in this Section and shall disclose the Disclosing Partys Confidential Information only to its Representatives who are legally bound to protect the Confidential Information under terms at least as restrictive as those provided herein. The Receiving Party agrees to protect the Disclosing Partys Confidential Information from unauthorized use or disclosure by exercising at least the same degree of care it uses to protect its own similar information, but in no event less than a reasonable degree of care. The Receiving Party shall be liable to the Disclosing Party for any disclosure or other breach in violation of this Agreement by any of its Representatives. The Receiving Party shall promptly notify the Disclosing Party of any known unauthorized use or disclosure of the Disclosing Partys

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

339

Confidential Information and will cooperate with the Disclosing Party in any litigation brought by the Disclosing Party against third parties to protect its proprietary rights. 15. Usage Verification. At Quest's request, but not more frequently than once per year, Customer shall furnish Quest with a document signed by an authorized representative verifying Customer's installations and usage of the Products. Customer will permit Quest to review Customer's deployment and use of the Products for compliance with the terms and conditions of this Agreement. Any such reviews shall be scheduled at least fifteen (15) days in advance, shall be conducted during normal business hours at Customer's facilities, and shall not unreasonably interfere with Customer's business activities. If Customer's use of the Products is found to be greater than contracted for Customer will be invoiced for the additional use and the unpaid fees shall be payable in accordance with this Agreement. Additionally, if the unpaid fees exceed five percent (5%) of the fees paid for the subject Products, then Customer shall also pay Quest's reasonable costs of conducting the audit. This Section shall not limit or restrict any other rights or remedies of Quest that are otherwise set forth in this Agreement or available at law. 16. General. (a) Governing Law and Venue. This Agreement shall be governed by and construed in accordance with the laws of the State of California, without giving effect to any conflict of laws principles that would require the application of laws of a different state. The parties agree that neither the United Nations Convention on Contracts for the International Sale of Goods, nor the Uniform Computer Information Transaction Act (UCITA) shall apply to this Agreement, regardless of the states in which the parties do business or are incorporated. Any action seeking enforcement of this Agreement or any provision hereof shall be brought exclusively in the state or federal courts located in the County of Orange, State of California, United States of America. Each party hereby agrees to submit to the jurisdiction of such courts. (b) Assignment. Except as otherwise set forth herein, Customer shall not, in whole or part, assign or transfer any part of this Agreement or any rights hereunder without the prior written consent of Quest. Any attempted transfer or assignment by Customer that is not permitted by this Agreement shall be null and void. (c) Severability. If any provision of this Agreement shall be held by a court of competent jurisdiction to be contrary to law, such provision will be enforced to the maximum extent permissible and the remaining provisions of this Agreement will remain in full force and effect. Notwithstanding the foregoing, the terms of this Agreement that limit, disclaim, or exclude warranties, remedies or damages are intended by the parties to be independent and remain in effect despite the failure or unenforceability of an agreed remedy. The parties have relied on the limitations and exclusions set forth in this Agreement in determining whether to enter into it. (d) Use by U.S. Government. The Software is a commercial item under FAR 12.201. Consistent with FAR section 12.212 and DFARS section 227.7202, any use, modification, reproduction, release, performance, display, disclosure or distribution of the Software or Documentation by the U.S. government shall be governed solely by the terms of this Agreement and shall be prohibited except to the extent expressly permitted herein. (e) Personal Data. Customer hereby acknowledges and agrees that Quests performance of this Agreement may require Quest to process or store personal data of Customer, its employees and Affiliates and to transmit such data internally within Quest or to Quest Affiliates. Such

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

340

processing, storage, and transmission shall only be to the extent necessary for, and for the sole purpose of, enabling Quest to perform its obligations under this Agreement and may take place in any of the countries in which Quest and its Affiliates conduct business, which may include countries outside of the European Economic Area. Quest hereby affirms to Customer that Quest Software, Inc. currently abides by the safe harbor framework as set forth by the U.S. Department of Commerce regarding the collection, use and retention of data from the European Union. (f) Notices. All notices provided hereunder shall be in writing, delivered personally, sent by facsimile or e-mail, or mailed by first class mail, postage prepaid, addressed to the legal department of the respective party or to such other address as may be specified in an Order or in writing by either of the parties to the other in accordance with this Section. All notices, requests, demands or communications shall be deemed effective upon personal delivery or four (4) days following deposit in the mail in accordance with this paragraph. (g) Disclosure of Customer Status. Quest may include Customer in its listing of customers and, upon written consent by Customer, announce Customer's selection of Quest in its marketing communications. (h) Waiver. Performance of any obligation required by a party hereunder may be waived only by a written waiver signed by an authorized representative of the other party, which waiver shall be effective only with respect to the specific obligation described therein. Any waiver or failure to enforce any provision of this Agreement on one occasion will not be deemed a waiver of any other provision or of such provision on any other occasion. (i) Injunctive Relief. Each party acknowledges and agrees that in the event of a material breach of this Agreement, including but not limited to a breach of the Software License," Restrictions or "Nondisclosure" Sections of this Agreement, the non-breaching party shall be entitled to seek immediate injunctive relief, without limiting its other rights and remedies. (j) Force Majeure. Each party will be excused from performance for any period during which, and to the extent that, it is prevented from performing any obligation or service as a result of causes beyond its reasonable control, and without its fault or negligence, including without limitation, acts of God, strikes, lockouts, riots, acts of war, epidemics, communication line failures, and power failures. Nothing in the foregoing shall be deemed to relieve Customer or its Affiliates of its obligation to pay fees owed under this Agreement. (k) Equal Opportunity. Quest Software Inc. is a federal contractor and Affirmative Action employer (M/F/D/V) as required by the Equal Opportunity clause C.F.R. 60-741.5(a). (l) Headings. Headings in this Agreement are for convenience only and do not affect the meaning or interpretation of this Agreement. This Agreement will not be construed either in favor of or against one party or the other, but rather in accordance with its fair meaning. When the term including is used in this Agreement it will be construed in each case to mean including, but not limited to. (m) Entire Agreement. This Agreement is intended by the parties as a final expression of their agreement with respect to the subject matter hereof and may not be contradicted by evidence of any prior or contemporaneous agreement unless such agreement is signed by both parties. In the absence of such an agreement, this Agreement shall constitute the complete and exclusive statement of the terms and conditions and no extrinsic evidence whatsoever may be introduced

Benchmark Factory for Databases User Guide Copyright, Licensing, and Licensing Agreement

341

in any judicial proceeding that may involve the Agreement. In the event of a conflict between the terms of this Agreement and the terms contained in an Order, the terms in the Order shall only control if the Order is signed by both Quest and Customer; otherwise, the terms of this Agreement shall control. Neither this Agreement, nor an Order, may be modified or amended except by a writing executed by a duly authorized representative of each party. No other act, document, usage or custom shall be deemed to amend or modify this Agreement or an Order. Delivery of Products shall be FOB Shipping Point.

Appendix:Contact Quest
Contact Quest/About Quest Software
About Quest
Quest Software simplifies and reduces the cost of managing IT for more than 100,000 customers worldwide. Our innovative solutions make solving the toughest IT management problems easier, enabling customers to save time and money across physical, virtual and cloud environments. For more information about Quest go to www.quest.com.

Contacting Quest Software

Email info@quest.com Quest Software, Inc. World Headquarters Mail 5 Polaris Way Aliso Viejo, CA 92656 USA Web site www.quest.com

See our web site for regional and international office information.

ContactingQuest Support
Quest Support is available to customers who have a trial version of a Quest product or who have purchased a Quest product and have a valid maintenance contract. Quest Support provides unlimited 24x7 access to SupportLink, our self-service portal. Visit SupportLink at http://support.quest.com. From SupportLink, you can do the following: Retrieve thousands of solutions from our online Knowledgebase Download the latest releases and service packs Create, update and review Support cases View the Global Support Guide for a detailed explanation of support programs, online services, contact information, policies and procedures. The guide is available at: http://support.quest.com.

Create

189

Index
A Advanced Advanced Creation Objects Agent Agents Overview AS3AP Benchmark B Benchmark Factory Components 32-34, 36, 50, 59, 76, 83, 121, 176, 207 Benchmark Factory Views Benchmark Object Wizard 38, 40 79, 150 79 79 134 33 223

Create Execute prepared SQL Statement Transaction193 Create_Delete_Benchmark Creation Custom load scenario D Data Database Quick Start E Environment 19 317 293 52, 57 79 170

Environment#Understanding_Benchmarks20 Excel 221

Excel-Command Line Exporting of Test Results199 Execute File Wizard Exponential Exporting Exporting Test Results to Excel F 83, 153 66 221 221

Benchmarks 18-19, 79, 150, 222-226, 229, 231, 310 BFCreditCardExp BFFileArray BFFormat BFMaxNode BFNumberOfIterations BFState BFTrimRight BFURandRange BFUserID C 236 239 246 260

Factory 261 245 259 249 264 Getting Started Global Search and Replace Globally Search Goal Test Goal Testing Quick Start Clustering Command Line Wizard 149 317 G

19-20, 310

17 93, 104 93, 104 172 94, 300

Benchmark Factory for Databases User Guide Index

344

H How Do I... Create a Load Scenario I 123, 154 Objects

O 79 317 320 286 40 198 P

Oracle's Instant Client Oracle Trace Files Oracle Trace Import-Database Replay

Import Stored Procedure Iteration Overruns J Job Job Wizard L Latency Latency_Keying Latency_Uniform Load Scenario Wizard

162, 187 Output View 61, 203 Overview

39, 55, 116 Prepare 74, 184 Preview Profile Creation Wizard 66 65 66 155-157, 159, 162164, 166-172, 174, 187 310 Profiles Project Node Project View Properties Overview Properties View Q Queue#Jobs_Queue_Toolbar 317 170 310 R N Replay Test 171 34, 50, 59, 76, 207 36 198, 221 106, 267 Quick Quick Starts 39 86, 179 94, 106, 267, 278, 286, 293, 300 92 72, 175 189

72, 74, 142-143, 175 41 38 41, 55 40

Local Area Connections M Migrating Mix MySQL Initialization Settings

Negative New Node

66 Reports 18 Repository Manager 41, 52, 55, 57, 63, 205 Results rpose of this Quic

Benchmark Factory for Databases User Guide Index

345

Run Run Reports Run Reports Viewer

34, 50, 59, 76, 92, 207 34, 50, 59, 76, 207 212

Time TPC-B Benchmark TPC-C Benchmark

65 225 226 229 231 189 186 U

Running Benchmark Factory with Multiple Agents121 TPC-D Benchmark TPC-H Benchmark S Transaction Scalable Hardware Benchmark Schedule Script Script Properties 224 Transaction Wizard 116 41 34, 41-43, 46-48, 50, 52, 55-57, 59, 76, 207 83, 176, 234 310 119 Upgrading Your Database Server User Scenario Wizard Using Using Run Reports W SQL Server 2005 Files Stagger User Startup 310 What's 120 Wizards Start#Entering_the_desired_SQL_statement 86, 179 Statement Store Procedure Examples Submit Job Wizard Summary System Requirements T Test Test Results Test Results Overview 159, 170, 198, 221 221 198 189 162, 187, 318 74, 184 63, 205 322 Understanding Understanding Benchmark Factory

20 19, 21, 24-25 278

Scripts Overview Server

Set Number of Users on an Agent Settings SQL

43, 195-197 310, 317 214

125, 127-129, 131, 133-134 86, 92, 159, 179, 189, 310 86, 159, 179

SQL Scalability Wizard

18 80, 83, 123, 150-154

ColleagueR18InstallationProcedures 02212013
No ratings yet
ColleagueR18InstallationProcedures 02212013
312 pages
SQL Project With Codings
No ratings yet
SQL Project With Codings
53 pages
SCCM
No ratings yet
SCCM
1,900 pages
XAF Application From Scratch
25% (4)
XAF Application From Scratch
18 pages
3D OCT 1 User Manual Version 8.1X PDF
No ratings yet
3D OCT 1 User Manual Version 8.1X PDF
236 pages
DAC
No ratings yet
DAC
414 pages
Qm852 Troubleshooting Guide Cisco
No ratings yet
Qm852 Troubleshooting Guide Cisco
90 pages
Oracle Business Intelligence Enterprise Edition Deployment Guide
No ratings yet
Oracle Business Intelligence Enterprise Edition Deployment Guide
206 pages
OPTIMA Installation Tool
No ratings yet
OPTIMA Installation Tool
76 pages
Core Infrastructure SCCM Side
No ratings yet
Core Infrastructure SCCM Side
1,875 pages
At G Pub Admin Guide
No ratings yet
At G Pub Admin Guide
313 pages
1.eBillingInstall 6.0
No ratings yet
1.eBillingInstall 6.0
104 pages
Siebel Data Quality Administration Guide: Version 8.0, Rev. C July 2011
No ratings yet
Siebel Data Quality Administration Guide: Version 8.0, Rev. C July 2011
202 pages
EAI2
No ratings yet
EAI2
316 pages
Oracle Business Intelligence Data Warehouse Administration Console Guide
No ratings yet
Oracle Business Intelligence Data Warehouse Administration Console Guide
128 pages
Introduction To IBM Rational Performance Tester V7.0
No ratings yet
Introduction To IBM Rational Performance Tester V7.0
19 pages
Introduction To Windows Azure
No ratings yet
Introduction To Windows Azure
29 pages
SOASTACloudTest Performance Testing Tutorial
No ratings yet
SOASTACloudTest Performance Testing Tutorial
52 pages
OBIEE Install
No ratings yet
OBIEE Install
260 pages
Sybase Performance and Tuning Guide-Vol2
No ratings yet
Sybase Performance and Tuning Guide-Vol2
380 pages
Getting Started With Stat Web Services API
No ratings yet
Getting Started With Stat Web Services API
18 pages
EAI3
No ratings yet
EAI3
214 pages
OrgPublisher 10 1 Installation Guide
No ratings yet
OrgPublisher 10 1 Installation Guide
53 pages
Stat 5.8.0 Upgrade Notes
No ratings yet
Stat 5.8.0 Upgrade Notes
37 pages
Cognos Admin Strati On and Security
No ratings yet
Cognos Admin Strati On and Security
400 pages
FT Viewpoint Quick Start
No ratings yet
FT Viewpoint Quick Start
46 pages
Hexagon Documentation installation guide
No ratings yet
Hexagon Documentation installation guide
256 pages
Starfish Installation & Usage Guide
No ratings yet
Starfish Installation & Usage Guide
71 pages
Oracle9i JDeveloper - Develop Applications With BC4J
No ratings yet
Oracle9i JDeveloper - Develop Applications With BC4J
454 pages
Configuration Guide: Smartconnector™ For Oracle Audit DB
No ratings yet
Configuration Guide: Smartconnector™ For Oracle Audit DB
21 pages
Install Admin
No ratings yet
Install Admin
304 pages
Steps To Install SQL Server 2022 On Windows Operating Systems To Log FactoryTalk Alarms and Events
No ratings yet
Steps To Install SQL Server 2022 On Windows Operating Systems To Log FactoryTalk Alarms and Events
18 pages
QuestSQLOptimizerForOracle UserGuide PDF
No ratings yet
QuestSQLOptimizerForOracle UserGuide PDF
87 pages
Toad Data Modeler 4.0.6.15 Freeware Installation Guide
No ratings yet
Toad Data Modeler 4.0.6.15 Freeware Installation Guide
21 pages
Odi Rel Note 10g
No ratings yet
Odi Rel Note 10g
8 pages
Forte For Java, Enterprise Edition, 3.0
No ratings yet
Forte For Java, Enterprise Edition, 3.0
124 pages
Viewmanager3 Admin Guide
No ratings yet
Viewmanager3 Admin Guide
182 pages
MicrosoftTFSVersionControlInAX5_wp
No ratings yet
MicrosoftTFSVersionControlInAX5_wp
22 pages
Web Service
0% (1)
Web Service
84 pages
View 51 Integration
No ratings yet
View 51 Integration
80 pages
Toad UserGuide
No ratings yet
Toad UserGuide
348 pages
Spotlight On Oracle Getting Started
No ratings yet
Spotlight On Oracle Getting Started
115 pages
How To Update Console Extentions
No ratings yet
How To Update Console Extentions
1,968 pages
11g Audit Vault
No ratings yet
11g Audit Vault
47 pages
6426B: Configuring and Troubleshooting Identity and Access Solutions With Windows Server 2008 Active Directory
No ratings yet
6426B: Configuring and Troubleshooting Identity and Access Solutions With Windows Server 2008 Active Directory
36 pages
VantagePoint Getting Results Guide PDF
No ratings yet
VantagePoint Getting Results Guide PDF
22 pages
FT Historian SE Installation Assistant
No ratings yet
FT Historian SE Installation Assistant
26 pages
Vcenter Orchestrator 551 Developers Guide PDF
No ratings yet
Vcenter Orchestrator 551 Developers Guide PDF
164 pages
8.2 Admin Guide
No ratings yet
8.2 Admin Guide
400 pages
Visual SourceSafe 2005 Software Configuration Management in Practice
From Everand
Visual SourceSafe 2005 Software Configuration Management in Practice
Aleksandar Seovic
No ratings yet
WebSphere Application Server 7.0 Administration Guide
From Everand
WebSphere Application Server 7.0 Administration Guide
Steve Robinson
No ratings yet
Oracle SOA Suite 11g Administrator's Handbook
From Everand
Oracle SOA Suite 11g Administrator's Handbook
Ahmed Aboulnaga
No ratings yet
Mastering GeoServer
From Everand
Mastering GeoServer
Colin Henderson
No ratings yet
Learning VMware vRealize Automation: Learn the fundamentals of vRealize Automation to accelerate the delivery of your IT services
From Everand
Learning VMware vRealize Automation: Learn the fundamentals of vRealize Automation to accelerate the delivery of your IT services
Sriram Rajendran
No ratings yet
Professional Microsoft SQL Server 2008 Administration
From Everand
Professional Microsoft SQL Server 2008 Administration
Brian Knight
No ratings yet
VMware vRealize Orchestrator Essentials: Get hands-on experience with vRealize Orchestrator and automate your VMware environment
From Everand
VMware vRealize Orchestrator Essentials: Get hands-on experience with vRealize Orchestrator and automate your VMware environment
Daniel Langenhan
No ratings yet
Implementing VMware Horizon View 5.2
From Everand
Implementing VMware Horizon View 5.2
Jason Ventresco
No ratings yet
Oracle SOA BPEL Process Manager 11gR1 A Hands-on Tutorial
From Everand
Oracle SOA BPEL Process Manager 11gR1 A Hands-on Tutorial
Ravi Saraswathi
5/5 (1)
VMware Horizon View Essentials
From Everand
VMware Horizon View Essentials
Peter von Oven
No ratings yet
Microsoft Forefront UAG 2010 Administrator's Handbook
From Everand
Microsoft Forefront UAG 2010 Administrator's Handbook
Erez Ben-Ari
No ratings yet
Microsoft Power Pages in Action: Accelerate your low-code journey with functional-rich web apps using Power Pages
From Everand
Microsoft Power Pages in Action: Accelerate your low-code journey with functional-rich web apps using Power Pages
Faisal Hussona
No ratings yet
Microsoft Application Virtualization Advanced Guide
From Everand
Microsoft Application Virtualization Advanced Guide
Augusto Alvarez
No ratings yet
Middleware Management with Oracle Enterprise Manager Grid Control 10g R5
From Everand
Middleware Management with Oracle Enterprise Manager Grid Control 10g R5
Arvind Maheshwari
3/5 (1)
Amazon RDS Migration
No ratings yet
Amazon RDS Migration
350 pages
Intro To
No ratings yet
Intro To
53 pages
Fundamentals of SQL Server 2012 Replication
No ratings yet
Fundamentals of SQL Server 2012 Replication
359 pages
VenkateswaraReddy Sqldba 4.2 Yrs
No ratings yet
VenkateswaraReddy Sqldba 4.2 Yrs
4 pages
Artificial Intelligence CW2 - Chatbot
No ratings yet
Artificial Intelligence CW2 - Chatbot
30 pages
Suresh .Net Developer
No ratings yet
Suresh .Net Developer
3 pages
Connect Microsoft Fabric Lakehouse With SQL Server Management System (SSMS)
No ratings yet
Connect Microsoft Fabric Lakehouse With SQL Server Management System (SSMS)
9 pages
007-001478-001 SAS PCE SP1 CRN 3.15 CRN RevD
No ratings yet
007-001478-001 SAS PCE SP1 CRN 3.15 CRN RevD
8 pages
Power BI Interview Questions
No ratings yet
Power BI Interview Questions
55 pages
AppSync 4.0 Security Configuration Guide
No ratings yet
AppSync 4.0 Security Configuration Guide
36 pages
Summary:: Technical Skills: Rdbms Operating Systems
No ratings yet
Summary:: Technical Skills: Rdbms Operating Systems
6 pages
Presentation PDF
100% (1)
Presentation PDF
44 pages
SAP CPI Questions and Answers - Ambikeya
No ratings yet
SAP CPI Questions and Answers - Ambikeya
17 pages
SQL Server Stored Procedures
No ratings yet
SQL Server Stored Procedures
10 pages
Rakesh Power Bi.g
No ratings yet
Rakesh Power Bi.g
4 pages
Fruit Shop Management System Project Report
No ratings yet
Fruit Shop Management System Project Report
44 pages
Peter Lalovsky Learn Microsoft SQL Server Intuitively. Transact SQL The Solid Basics
100% (1)
Peter Lalovsky Learn Microsoft SQL Server Intuitively. Transact SQL The Solid Basics
289 pages
DBA - Alex Bingao
No ratings yet
DBA - Alex Bingao
3 pages
OMNITREND Center Installation LIT 82.201 012018 en
No ratings yet
OMNITREND Center Installation LIT 82.201 012018 en
30 pages
Gps Tracker
No ratings yet
Gps Tracker
7 pages
Postgraduate Diploma in Information and Communication Technology
No ratings yet
Postgraduate Diploma in Information and Communication Technology
11 pages
SecurityCmdLnApp LogFile
No ratings yet
SecurityCmdLnApp LogFile
2 pages
Puja Pallavi Resume Old
No ratings yet
Puja Pallavi Resume Old
2 pages
DAP_Module3
No ratings yet
DAP_Module3
42 pages
Monish KT Resume
No ratings yet
Monish KT Resume
2 pages
DMS C Final
No ratings yet
DMS C Final
22 pages
Snigdha Nalubolu Dot Net
No ratings yet
Snigdha Nalubolu Dot Net
11 pages