Ims12 workbench data visualization - IMS UG May 2014 Sydney & Melbourne

© 2013 IBM Corporation©2014 IBM Corporation
IBM Transaction Analysis Workbench
Version 1.2
IMS-DB2 and more

© 2013 IBM Corporation©2014 IBM Corporation©2014 IBM Corporation
Agenda
1. Introduction: issues of problem solving today
2. Solution: IBM Transaction Analysis Workbench
3. DB2 exception reporting and extract
4. Interactive log analysis – signature feature
5. ISPF dialog – plus Eclipse GUI for first responder
6. Session Manager – collaborative problem solving
7. Applications development
8. IMS DB2 scenario
9. SMF reports
10.CICS-DBCTL1

Applications have evolved
2
1980s application:
in-house users only; simple data, single data store
Today:
users are customers; data is complex, heterogeneous, often distributed
Transaction Manager:
IMS DC or CICS
Users:
company
employees
Database:
IMS DB or DB2
Users:
employees,
customers (public),
automated systems
Transaction Manager:
IMS DC or CICS
WebSphere MQ
IMS DB
DB2
non Z servers

DB2
MQ
DLI
Where did the delay occur?
 A single transaction can
have activity across
many subsystems
 To quickly identify
performance issues,
you need to track the
entire transaction
 Subsystem-specific
approaches and tools
offer a limited
perspective
 Each subsystem has its
own activity log and
SMF records
3
IMS

Traditional approaches and tools based on
“silo” model
4
There are many tools to help analyze individual transaction
environments on System z:
One
transaction,
not four!
CICS
Performance
Analyzer
IMS
Performance
Analyzer
DB2
Performance
Expert
Many others...
CICS IMS DB2
CICS
IMS
DB2
MQ
One
transaction,
not three!
IMS
DB2
MQ
Each tool is well-suited
to its environment, but
SME uses their own
tools

IBM Transaction Analysis Workbench for z/OS
 A tool for problems in the big picture:
– For “first responders” and subject-matter experts (SMEs)
– For SMEs in different areas
 Provides a life cycle view of transaction activity across
subsystems
– Changes the way problem resolution is performed
– Ensures everyone is looking at the same transactional data
 Goes in-depth.
– Uses SMF, trace, and log records to follow transaction flow
 Better assignment of problems to the correct group
– Improved confidence in problems assigned to experts
5

Coverage: many subsystems, many logs
IBM Confidential
IMS CICS DB2
WebSphere
MQ, App Server z/OS
IMS log and trace
CMF performance
class (SMF 110)
DB2 log MQ log extract
Various SMF
record types
IMS monitor CICS trace
DB2 trace
(IFCIDs)
MQ statistics
(SMF 115-1, -2)
OPERLOG
CQS log stream
MQ accounting
(SMF 116)
IMS Connect event
data (collected by
IMS Connect
Extensions)
WAS request
activity
performance
statistics
(SMF 120-9)
OMEGAMON
ATF
IRLM long lock
detection
(SMF 79-15)

Cross support is key with fewer staff
 Why is support so difficult?
– SME may be in silos
• Cross training may be difficult
• Takes too much time using current tools
• Unable or unwilling to cooperate easily
 What does good cross-platform tool achieve?
– Conservation of SMEs’ time (a valuable, limited resource)
– Transparency of information (everyone using the correct data
from the same time period)
 SMEs need to see the big picture and the benefit of
collaboration
– Reduced time to resolution
– More SME time focused on problem resolution
– Cross-training of first responders and SMEs7

What’s new in Workbench V1.2
 CICS-DB2 and IMS-DB2 transaction exception processing
Was the problem in CICS or DB2? IMS or DB2? (CICS-DBCTL already
supported in V1.1.)
 Enhanced support for DB2 trace records
Detailed field-by-field formatting for more than 60 IFCIDs.
 Workflows and session templates
Subject-matter experts (SMEs) can define a workflow (a sequence of
analysis tasks) and save it in a session template. When creating a
new session, users can select the session template that best
matches the report problem.
 Eclipse-based rich client platform (RCP) user interface
Implements a subset of the ISPF dialog: create a session; run a
workflow; assign to appropriate SME.
 Automated SMF file selection
 SMF 42.6 DASD Data Set I/O report
8

Problem: today’s instrumentation data overwhelms
traditional tools
 Good performance monitoring should identify possible performance
issues before they become critical
 Today's systems create so much instrumentation data that existing
techniques cannot keep up: takes too long, costs too much!
9
Instrumentation data
Report
Traditional batch
analysis/reporting tool
• Processes and reports on all
records
• Processing time and cost
grows with size of
instrumentation data,
beyond practical limits
• Reports can grow too long to
be useful, and contain
unwanted detail

Solution: Exception detection
10
Terabytes of instrumentation data
Workbench
Transaction
index
Exception: a transaction that matches specific exception
criteria, such as long response time or an abend
z/OS batch
process that
efficiently crawls
data for
exceptions
Reporting
and
interactive
analysis on
reduced
data

Solution: Transaction Index
11
CICS performance
records
SMF 110
CICS
Transaction Index
DB2 accounting
records
SMF 101, IFCID 3
DB2
Transaction Index
IMS log
IMS
Transaction
Index
WebSphere MQ
accounting
records
SMF 116
WebSphere MQ
Transaction
Index
Workbench exception processing
1. Transaction indexes are created by the workbench (a session workflow will create them)
2. They are used to identify all the transaction and UOR workloads in IMS, DB2 and CICS
3. The transaction index is a special extract - one record per transaction in time sequence
4. Contain summarized performance and resource usage information
5. Can be filtered to include exception transactions only
6. Can be used for reporting and to identify problem transactions

DB2 exception extract and reporting (DB2X)
 The DB2 exception extract and reporting facility helps to identify
and analyze problems associated with DB2.
 DB2 accounting records (SMF 101) are checked against user-
specified exception criteria. DB2 threads that are identified as
exceptions are reported and extracted for further analysis.
 The detailed list report lists those DB2 accounting records that
triggered an exception – with the reason
 The extract writes those DB2 accounting records that triggered an
exception to an extract data set. This data set can then be used for
more specialized analysis, for example:
o Tivoli OMEGAMON XE for DB2 Performance Expert
o Transaction Analysis Workbench ISPF dialog interactive
analysis; merged with the DB2 log and other traces to
examine the transaction lifecycle picture.
12

Response and CPU time: / Apportion rollup
Response time . . 0.5 (0.000001-99 seconds)
CPU class 1 . . . 0.1
In-DB2 elapsed . . 0.2
CPU class 2 . . . 0.05
Database I/O . . . 0.05
Lock suspend . . . 0.05
Stored Procedure:
Elapsed . . . . . 0.1 (0.000001-99 seconds)
CPU . . . . . . . 0.05
Row activity:
Fetched . . . . . 1000 (0 to 999999)
Inserted . . . . . 100
Updated . . . . . 100
Deleted . . . . . 100
Locking:
Deadlocks . . . . 1 (0 to 999999)
Suspends . . . . . 20
Timeouts . . . . . 1
Lock requests . . 50
Buffering:
Get pages . . . . 50 (0 to 999999)
Update pages . . . 30
Logging:
Log records . . . 100 (0 to 999999)
Abnormal conditions:
Abort . . . . . . 1 (1=check for condition)
Check pending . . 1
DB2X: Specify exception thresholds
13
DB2 threads that
exceed one of more
of these thresholds
are considered
exceptions

DB2X: List of DB2 exceptions
V1R2M0 2014-02-20 Thursday DB2 Exception List Page 1
SSID Correlation Connect Plan Auth id Time Exception Threshold LUWID
---- ------------ -------- -------- -------- --------------- ----------------------- --------- -----------------------------
DBA4 MQP1DB2SRV02 RRSAF CSQ5L710 STC@ZOSN 10:02:04.512982 Abort 1 1 FTS1/DBA4LU/CCBDD2D0C2CD/0002
DBP4 AXSSIGNO DB2CALL PTS46 PROTEUS 10:02:07.263130 Response 1.402972 0.5 FTS1/DBP4LU/CCBDD2D20BE5/0005
DBP4 AXSSIGNO DB2CALL PTS46 PROTEUS 10:02:07.263130 Class 1 CPU 0.387519 0.1 FTS1/DBP4LU/CCBDD2D20BE5/0005
DBP4 AXSSIGNO DB2CALL PTS46 PROTEUS 10:02:07.263130 Rows fetched 6329 1000 FTS1/DBP4LU/CCBDD2D20BE5/0005
DBP4 AXSSIGNO DB2CALL PTS46 PROTEUS 10:02:07.263130 Lock requests 267 50 FTS1/DBP4LU/CCBDD2D20BE5/0005
DBP4 AXSSIGNO DB2CALL PTS46 PROTEUS 10:02:07.263130 Get pages 2264 50 FTS1/DBP4LU/CCBDD2D20BE5/0005
DBP4 AXSSIGNO DB2CALL PTS46 PROTEUS 10:02:07.263130 Update pages 483 30 FTS1/DBP4LU/CCBDD2D20BE5/0005
DBP4 AXSSIGNO DB2CALL PTS46 PROTEUS 10:02:07.263130 Log records 574 100 FTS1/DBP4LU/CCBDD2D20BE5/0005
DBP4 MXMSCHD DB2CALL DSNREXX MXM 10:08:44.907535 Get pages 2668 50 FTS1/DBP4LU/CCBDD44E91C7/0002
DBP4 MXMSCHD DB2CALL DSNREXX MXM 10:08:46.466276 Get pages 2668 50 FTS1/DBP4LU/CCBDD4500BE1/0002
DBA4 MQP1DB2SRV02 RRSAF CSQ5L710 STC@ZOSN 10:10:04.877879 Abort 1 1 FTS1/DBA4LU/CCBDD49ADF64/0002
DBP4 AXS#GENE DB2CALL PTS46 PROTEUS 10:10:05.766411 Response 1.338092 0.5 FTS1/DBP4LU/CCBDD49A71E6/0011
DBP4 AXS#GENE DB2CALL PTS46 PROTEUS 10:10:05.766411 Class 1 CPU 0.361788 0.1 FTS1/DBP4LU/CCBDD49A71E6/0011
DBP4 AXS#GENE DB2CALL PTS46 PROTEUS 10:10:05.766411 Lock requests 135 50 FTS1/DBP4LU/CCBDD49A71E6/0011
DBP4 AXS#GENE DB2CALL PTS46 PROTEUS 10:10:05.766411 Get pages 223 50 FTS1/DBP4LU/CCBDD49A71E6/0011
DBP4 AXS#GENE DB2CALL PTS46 PROTEUS 10:10:05.766411 Update pages 72 30 FTS1/DBP4LU/CCBDD49A71E6/0011
14
For each exception that was triggered, the thread and exception details are listed
SSID DB2 subsystem id
Correlation Correlation id
Connect Connection name. If the name is not available then the connecting system type is substituted
Plan Plan name
Auth id Authorization id
Time End of accounting interval stamp. This is not the thread start time.
This time matches the record timestamp displayed in the ISPF dialog.
Exception The exception event that was triggered – its description and value
Threshold The exception threshold that was exceeded
LUWID Logical unit of work id, used to uniquely identify the DB2 accounting record.

Interactive Log analysis using the ISPF dialog
 Allows you to view transactions events across all the
subsystems
 Merges data sources from IMS, DB2, CICS and MVS
 Displays records with friendly descriptions and useful
information
 Analyze transaction event latencies with relative and
elapsed times
 Use a filter to show required records only
 Use tracking to isolate all the records associated with
an individual transaction
 Select a record to see more detail (full mapping)15

IMS-DB2 transaction Part 1
16
BROWSE FUW000.QADATA.FBOSPM4C.IMS.D130703.INDEX Record 00005529 More: < >
Command ===> Scroll ===> CSR
Navigate < 00.00.01.000000 > Date/Time 2013-07-03 16.33.53.849552
/ ———— Tracking ———————————————————————— Wednesday 2013-07-03 Time (Relative)
CA01 IMS Transaction TranCode=FBOICT42 Region=0002 16.40.55.075503
01 Input Message TranCode=FBOICT42 +0.000000
31 DLI GU TranCode=FBOICT42 Region=0002 +0.000266
5600 Sign-on to ESAF Region=0002 SSID=DBA6 +0.005517
112 Thread allocate FBOIAP42 DBA6 +0.005996
177 Package allocation FBOIAP42 DBA6 +0.006304
380 SP entry FBOSPM4C STMT=001044 DBA6 +0.006511
177 Package allocation FBOSPM4C DBA6 +0.006681
060 SQL SELECT STMT=000009 DBA6 +0.006796
058 SQL call completion SQLCODE=0 STMT=000009 DBA6 +0.016090
061 SQL DELETE STMT=000011 DBA6 +0.016387
0020 Begin UR +0.017584
325 Trigger entry USERDEL STMT=000011 DBA6 +0.017762
177 Package allocation USERDEL DBA6 +0.017836
061 SQL INSERT STMT=000002 DBA6 +0.017887
Continued on next foil...
1. IMS transaction starts with an Index record (CA01)
2. IMS log (green), DB2 trace (yellow) and DB2 log (turquoise) events are shown
3. Transaction calls a DB2 stored procedure which issues SQL calls, call results are shown
IMS
DB2

17
... Continuation from previous foil
061 SQL DELETE STMT=000003 DBA6 +0.018179
0600 Delete from a data page . . . +0.019696
0600 Delete from a data page +1.082712
325 Trigger exit SQLCODE=0 DBA6 +1.083254
499 SP statement execution detail DBA6 +1.083370
380 SP exit FBOSPM4C SQLCODE=0 STMT=001044 DBA6 +1.083400
053 SQL request SQLCODE=0 STMT=001044 DBA6 +1.083436
5600 Commit Prepare starting Region=0002 SSID=DBA6 +1.084442
084 Prepare start DBA6 +1.084894
0020 End commit phase 1 +1.085008
085 Prepare end DBA6 +1.086509
03 Output Message Response LTerm=FUNTRM07 +1.086577
3730 Syncpoint End of Phase 1 Region=0002 +1.086598
074 Terminate thread start DBA6 +1.086969
0020 Begin commit phase 2 +1.087056
0020 End commit phase 2 +1.087728
003 Thread accounting DBA6 +1.089002
075 Terminate thread end DBA6 +1.089944
5600 Commit Continue completed Region=0002 SSID=DBA6 +1.089970
37 Syncpoint Message Transfer Region=0002 +1.089987
5612 Syncpoint End of Phase 2 Program=FBOIAP42 Region=0002 +1.090012
31 Communications GU LTerm=FUNTRM07 +1.090054
36 Output Message Dequeue LTerm=FUNTRM07 +1.093952
******************************* Bottom of Data ********************************
IMS
SYNCPOINT
DB2
Delay
4. SQL DELETE causes trigger to delete many more table rows, causing the delay

18
BROWSE FUW000.QADATA.FBOSPM3C.IMS.D131011.INDEX Record 00006876 More: < >
/ ———— Tracking ———————————————————————— Friday 2013-10-11 Time (LOCAL)
CA01 IMS Transaction 11.33.58.759345
TranCode=FBOIAT42 Program=FBOIAP42 Userid=FUNTRM14
LTerm=FUNTRM14 Terminal=SC0TCP14 SSID=IDDG Region=0001
RecToken=IDDG/0000000E00000000 OrgUOWID=IDDG/CC17F12AB6B85204
InputQ=0.000332 Process=1.094017 OutputQ=0.000028 TotalTm=0.134377
CPU=0.111167
————————————————————————————————————————————————————————————————————————————
. . .
————————————————————————————————————————————————————————————————————————————
003 DB2 Thread accounting DBA6 11.33.58.893342
TranCode=FBOIAP42 Program=FBOIAP42 Userid=FUNTRM14
RecToken=IDDG/0000000E00000000 ClientID=IDDG Region=0001
LUWID=FTS3/DBA6LU/CC17F12AB853/0002
RESP=1.086945 CPU1=0.107330 CPU2=0.000688 I/O3=0 Source=IMS_MPP
SEL=4 DEL=1 CAL=1 LogRecs=9777 GetPage=19777 UpdPage=9776 MaxLock=162
————————————————————————————————————————————————————————————————————————————
******************************* Bottom of Data ********************************
5. Scroll right to view the index record summary, CA01 is the IMS transaction
6. The DB2 accounting (IFCID 003) shows excessive logging, buffering and locking

+0120 QW0058ID... Scan information
+0120 Scan type.... 'SEQD' Rows processed... +24069
+0130 Rows examined.... +24069
+0138 Rows qualified after stage 1... +24069
+0140 Rows qualified after stage 2... +1
+0148 Rows inserted.... +0
+0150 Rows updated... +0
+0158 Rows deleted... +0
+0160 Pages scanned.... +428
+0164 Pages scanned (RI)... +0
+0168 Rows deleted (RI).... +0
+0170 Pages scanned (LOB).... +0
+0174 Pages updated (LOB).... +0
+01A0 QW0058TY... Statement-level information
+01A0 SQL statement type... 4000
+01A8 Statement ID... +28917
+01B0 Sync reads... +0 Getpages... +428
+01C0 Rows examined.... +24069
+01C8 Rows processed... +0 Sorts...... +0
+01D8 Index scans.... +0
+01E0 Table space scans.... +1
+01E8 Buffer writes.... +0
+01F0 Parallel groups.... +0
+01F8 In-DB2 elapsed... 0.008537
Enhanced support for DB2 trace (IFCID) records
 Detailed formatting of IFCID-specific fields
19
+0120 QW0058ID... 'SEQD' Scan type
Off QW0058IX... 'INDX' Index
On QW0058SD... 'SEQD' Sequential data
Off QW0058SW... 'SEQW' Sequential data workfile

Enhanced support for DB2 trace records (cont.)
 Generating DB2 trace records can be expensive, and can
result in very large log files: you do not want to simply start all
traces.
 Workbench introduces the concept of trace “levels” (1 - 4) that
categorize IFCIDs based on their usefulness (for transaction
analysis) and cost overhead:
– Program invocation
– SQL
– I/O
– All (caution: may result in high volumes of data)
 In the ISPF dialog, enter the command:
TRACE n
(n: 1 - 4) to show progressively more detail. TRACE 4 shows all
available trace records.20

Workbench ISPF dialog
1. Sessions: collaborative approach to problem determination
2. Controls: common place for shared definitions
3. Systems: define IMS, DB2, CICS and MVS systems – allows
workflows to perform automated file selection (locate the
required log files)
4. Process: ad hoc log file processing
21
File Help
——————————————————————————————————————————————————————————————————————————————
V1R2M0 Transaction Analysis Workbench - Primary Option Menu
Option ===>
0 Profile Customize your dialog profile
1 Sessions Analyze problems using the session manager
2 Controls Define record filtering and formatting controls
3 Systems Define the systems where transactions are processed
4 Process Analyze ad hoc log files
X Exit Quit the workbench
Session Repository . . . 'FUW120.SESSIONS' +

Session Manager
1. Register your problems
2. Allows users to collaborate - see work already done so far
22
Session Manager Row 1 of 17 More: < >
Command ===> NEW Scroll ===> CSR
NEW Register a new Session
/ Key Status Description
00000001 OPEN DB2: CICS read via SP doing table space scan
00000002 DONE DB2: CICS update via SP doing table space scan
00000003 DONE DB2: IMS tran generating cascade deletes
00000004 DONE DB2: IMS tran generating trigger deletes
00000005 OPEN DB2: Java update via SP waiting for WLM
00000006 DONE DB2: IMS tran calling inefficient DB2 SP
00000007 OPEN DB2: Java update via SP waiting for CPU
00000008 OPEN DB2: z/OS WAS Java app gens RI cascade deletes
00000009 OPEN DB2: z/OS WAS Java app gens trigger deletes
00000010 OPEN DB2: z/OS WAS Java app calling inefficient SP
00000011 OPEN CICS-DBCTL
00000012 OPEN CICS-DBCTL deadlock
00000014 OPEN DB2: Contention
******************************* Bottom of data ********************************

The session – the place for collaboration
1. Register problem details
2. Workflow contains all the tasks to be performed
3. Files contains the list of log files for this problem
4. Investigate provides interactive problem determination
5. History contains written notes and jobs about the problem
23
File Help
——————————————————————————————————————————————————————————————————————————————
Session 00000003
Option ===>
Description : DB2: IMS tran generating cascade deletes
1 Register Update the problem registration details
2 Workflow Perform the diagnostic tasks
3 Files Locate and manage the log files required for diagnosis
4 Reporting Run batch reports
5 Investigate Perform interactive log file analysis
6 History Review the problem history

Session registration details
1. What is the problem? Including short and long descriptions
2. When and where did the problem occur? Important for log selection!
3. Who is going to fix it?
24
Session Details
Key . . . . : 00000003
Description . DB2: IMS tran generating cascade deletes
Severity . . 3
Reference . . FUW-755 —— When problem occurred ———
Reported by . Tony YYYY-MM-DD HH.MM.SS.TH
Assigned to . John From 2013-10-11 11.15.00.00
Status . . . OPEN To 2013-10-11 11.40.00.00
Template . . + Zone LOCAL
Systems involved:
/ System + Type +
ICDG IMS
IDDG IMS
DBA6 DB2
FTS3 IMAGE
******************************* Bottom of data ********************************

Workflows and session templates
 SMEs can use session templates to populate new sessions with the
tasks needed to prepare the problem for evaluation
– Created sessions include: systems involved and a sequence of tasks (workflow)
for analyzing the problem
25
Session template
Systems
Workflow
Task
Task
…
Session
Systems
Workflow
Task
Task
…
When creating a
session, users can
select the session
template that best
matches the type
of problem
Each task is either
JCL for a batch
job, or a note
containing
instructions to the
user

Session: Workflow
1. Contains tasks – batch jobs and instructions
2. Batch jobs can locate log files, create extracts and transaction indexes, run reports
3. Can be predefined with a “template” set up by the expert
4. All jobs run under the session create a new task
5. Batch job output is saved and can be viewed here
26
Tasks Row 1 to 9
NEW Create a new task
AUTO Create file selection and extract tasks
SCHED Schedule all the tasks (or select required tasks only)
/ Task Status Description
1 CC 0000 Select IMS logs for IMS system IDDG
2 CC 0000 Select SMF files for DB2 system DBA6
3 CC 0000 Select DB2 logs for DB2 system DBA6
4 CC 0000 Create log extract for IMS system IDDG
5 CC 0000 Create SMF extract for DB2 system DBA4
6 CC 0000 Create log extract for DB2 system DBA6
7 CC 0000 Create index for IMS systems
8 DONE Please assign the problem to John, our DB2 expert.
? 9 CC 0000 DB2 performance and accounting analysis for system DBA6
******************************* Bottom of data ********************************Task Output Row 1 to 5
/ DDname StepName ProcStep Rec-Cnt Jobname JobID Max-RC
JESMSGLG JES2 32 JCH#TAWD JOB03728 CC 0000
JESJCL JES2 25
JESYSMSG JES2 91
SYSPRINT FUWBATCH 37
S DB2X FUWBATCH 418
*************************** Bottom of data *****************************

Session: Files
1. All log files and other data sets associated with the problem
are registered here
2. Files can be automatically located - IMS, DB2 and SMF
3. Manually enter other types of log files
4. Supports all data set types and log streams
27
Locate and Manage Log Files Row 1 of 8 More: < >
NEW Insert a new log file.
AUTO Run automated file selection to locate log files.
Log Files:
———— System ———— File
/ Exc Data Set Name Name Type Type
S IMS.IDDG.INDEX IDDG IMS EXTRACT
S IMS.IDDG.SLDS IDDG IMS LOG
* IMS.IDDG.TRACE IDDG IMS LOG
* IMS.IDDG.MONITOR IDDG IMS MON
S DB2.DBA6.ARCLOG DBA6 DB2 LOG
S SMF:IFASMF.FTS3.SMF FTS3 IMAGE SMF
******************************* Bottom of data ********************************
SMF log stream
IMS and DB2 logs

Session: Reporting
1. IMS Performance Analyzer – selected reports only
2. CICS Performance Analyzer – selected reports only
3. CICS-DBCTL – end-to-end from CICS into IMS
4. DB2 – exception reporting and extract
5. OPERLOG – MVS operations log (SYSLOG)28
Reporting
Option ===>
Select a reporting option then press Enter.
1 IMS Transaction and system analysis using IMS PA
2 CICS Transaction and system analysis using CICS PA
3 CICS-DBCTL Combined CICS and IMS analysis of transactions
4 SMF z/OS and subsystem analysis
5 DB2 DB2 accounting exception analysis
6 OPERLOG Sysplex operations log (SYSLOG)

Session: Investigate
1. Session log files are merged in time sequence
2. Often log files are very large. Use time slicing to process
required time period only – very quick!
3. Select all or some of the files only
29
Investigate Row 1 of 4 More: < >
—————————————— Time Slice (ON) ————————————————
Time Date Duration
HH.MM.SS.thmiju YYYY-MM-DD HH.MM.SS Zone Filter +
16.39.36.351066 2013-07-03 00.00.00 LOCAL
/
Type Start Time Date Duration Coverage
IMS 16.39.36.351066 2013-07-03 Wed 00.02.08 PARTIAL
IMS 16.35.26.490921 2013-07-03 Wed 00.06.42 PARTIAL
DB2 16.33.53.849552 2013-07-03 Wed 00.27.46 COMPLETE
DTR 16.39.03.904776 2013-07-03 Wed 00.02.55 PARTIAL
******************************* Bottom of data ********************************
Type Data Set Name Coverage
IMS IMS.IDDG.INDEX PARTIAL
IMS IMS.IDDG.SLDS PARTIAL
DB2 DB2.DBA6.ARCLOG COMPLETE
DTR SMF:IFASMF.FTS3.SMF PARTIAL
******************************* Bottom of data ********************************

Eclipse-based rich client platform (RCP) UI
30

How Workbench can help
application development
teams
31

Application releases must work and perform when
deployed
 Application teams perform validation testing during roll-out
– Is performance a part of validation testing?
– If performance validation is done, who does the validation?
– What criteria are used?
 Does the evaluation occur at the transaction level?
 What is the cost of performance validation testing?
 What is the cost of a failed roll-out due to poor
performance?
 Does system programming have time to help?
32

Inhibitors to instrumentation data usage by application
development teams
 Value of instrumentation data not known
– May not know what is available and how to use it
– Not a traditional development tool
 Do not know how to obtain the data or data access
not allowed
– May not have access to system parts
 Limited or no knowledge of tools that use
instrumentation data
 Limited access to system programmers’ time
– Reluctant to bother system programmers to get help
33

Solution - Value of instrumentation data to application
development teams
 Workbench demystifies instrumentation data:
– Allows application teams to evaluate
• Transaction response times
• Application database update patterns
– Allows application teams to diagnose application errors
and/or performance issues
– If SME help is needed, they can ask better questions of
the right SME
34

How Workbench helps
 Automates the collection of instrumentation data
– Application development teams do not have to acquire those skills
 Performs automated reporting of validation testing
– Includes reporting via CICS PA and/or IMS PA, in addition to its
own reports
 Analyses instrumentation data for performance exceptions
– Provides easy recognition of validation testing against expected
results
 Provides transaction life cycle views of transaction exceptions
– Identify what part of transaction is causing problem
 Saves results of each validation testing run
 Facilitates collaboration with system programmers and/or DBAs
for help with transaction exception diagnosis35

Summary of application team benefits
 Automate tasks often unfamiliar to application teams
– Data acquisition – get the data needed for problem analysis
– Autonomics – automated transaction analysis (life cycle)
– Reporting – basic reporting without tool-specific knowledge
 Enables collaboration with other experts
– Shared data approach
• DBA, system programmer provide assistance when needed
– Fewer SMEs may need to be involved
 Analysis of applications performance testing
– Exceptions process provides evaluation of validation runs
– Deeper transaction evaluation if exceptions reported
36

Scenario: IMS DB2 problem
1. On the following slides, we present an example
scenario: a user has reported a long transaction
response time for an IMS transaction performing DB2
updates
 The analysis is divided into two parts:
1. The first responder:
• Registers the problem in the Workbench session manager and
collects the log files
• Follows a process orientated script to assign problem to initial
expert
– Based on what is found
2. The subject-matter expert performs a “deep dive” on the
problem: reviewing the reports, and using interactive analysis
to identify the specific log records for the cause of the
problem
38

File Help
——————————————————————————————————————————————————————————————————————————————
Session Details Row 1 to 3 of 3
Key . . . . : 00000026
Description . IMS DB2 problem
Severity . . 4
Reference . . TICKET-127 —— When problem occurred ———
Reported by . JOHN YYYY-MM-DD HH.MM.SS.TH
Assigned to . JAMES From 2012-06-24 15.20.00.00
Status . . . OPEN To 2012-06-24 16.50.00.00
Template . . IMS+DB2 + Zone LOCAL
Systems involved:
/ System + Type +
IADG IMS
DB3A DB2
FTS1 IMAGE
******************************* Bottom of data ********************************
39
First responder: Creating a session
The template (set up by the expert) populates the system list (where the problem
occurred), as well as the workflow task list (preparatory jobs to select log files and create
extracts)
Create a session (main menu ► option 1 Sessions ► NEW).
Select the environment (template) where the problem occurred.

© 2013 IBM Corporation©2014 IBM Corporation40
Eclipse-based
GUI
1. Register a new problem; work on an existing problem
2. Execute the workflow to locate the required diagnostic data
3. Run reports; view the output

File Mode Filter Time Labels Options Help
——————————————————————————————————————————————————————————————————————————————
BROWSE IMPOT01.SESSION7.TRANIX + Record 00004609 More: < >
/ ———— ————————————————————————————————— Friday 2013-10-11 Time (LOCAL)
TX CA01 Transaction 16.33.33.575325
UTC=16.33.33.575316 TranCode=MQATREQ1 Program=MQATPGM Userid=FUNTRM15
LTerm=FUNTRM15 Terminal=SC0TCP15 Region=0004
OrgUOWID=IADG/C62D2CB467860940 IMSID=IADG IMSRel=101
RecToken=IADG/0000003600000000
CPU=0.041999 InputQ=0.000562 Process=0.497229
TotalTm=0.497791 RegTyp=MPP DBCalls=5
----------------------------------------------------------------------------
CA01 Transaction 16.33.59.157812
OrgUOWID=IADG/C62D2CCCCD3E6F81 IMSID=IADG IMSRel=101
RecToken=IADG/0000003A00000000
TotalTm=0.424921 RegTyp=MPP
----------------------------------------------------------------------------
41
Subject-matter expert: Exception candidate investigation
This display has been filtered to show IMS transaction index (CA01) records with a process
time of greater than 0.4 seconds. Enter TX to show records related to a transaction

——————————————————————————————————————————————————————————————————————————————
FUWPRBRF GXH.FUW.JCH1.FUW745.UPDATE.CICS.EXTRACT Record 00000001 More: < >
/ ———— Tracking ———————————————————————— Friday 2013-05-31 Time (Relative)
TX 6E13 CICS Transaction TranCode=FB66 Task=944 16.27.24.275202
086 Signon start DBA6 +0.003469
072 Create thread start DBA6 +0.003546
112 Thread allocate DBA6 +0.003805
073 Create thread end DBA6 +0.003830
053 SQL DESCRIBE/COMMIT/ROLLBAC SQLCODE=0 STMT=000158 DBA6 +0.004096
233 SP entry FBOSP006 STMT=000196 DBA6 +0.005104
015 Index scan begin DBA6 +0.005874
018 Scan end DBA6 +0.006097
055 SQL set current SQLID DBA6 +0.006188
060 SQL SELECT STMT=000344 DBA6 +0.006365
017 Sequential scan begin DBA6 +0.006478
006 Read I/O begin DBA6 +0.006582
007 Read I/O end DBA6 +0.006950
018 Scan end DBA6 +1.609979
061 SQL UPDATE STMT=000423 DBA6 +1.610336
017 Sequential scan begin DBA6 +1.610463
0020 DB2 Unit of Recovery Control - Begin UR +1.610733
0010 DB2 Savepoint +1.610733
0020 DB2 Update In-Place in a Data Page +1.610749
018 Scan end DBA6 +1.610771
233 SP exit FBOSP006 SQLCODE=0 STMT=000196 DBA6 +1.611397
42
Transaction life cycle investigation 1. Start tracking a
transaction
(here, a CICS
transaction)
2. See the
transaction life
cycle events
from the related
logs (here, an
SMF file and a
DB2 log),
merged
together with no
preparation
required
3. Notice the jump
in elapsed time
4. In this case, the
problem was
caused by a
table scan in a
DB2 stored
procedure.
A drill down of
the DB2 trace
was able to
determine this.

Detail DB2 event data view using forms view
********************************* Top of data *********************************
+018C Code... 058 SQL Call completion RC=0000 STMT=002896 DBA6
+0198 Date... 2012-11-21 Wednesday Time... 17.40.04.013647.813
Package
+0034 Location...... 'DB2ALOC' Collection ID........ 'CSQ5L710'
+0056 Package name... 'CSQ5L710' Consistency token.... 193153A81425EA0D
+0072 SQLCA...... SQL communication area (SQLCA)
+0072 SQLCAID.... 'SQLCA ' SQLCABC.... +136 SQLCODE.... +0
+0082 SQLERRML... +0 SQLERRM.... ' '
+00CA SQLERRP.... 'DSN ' SQLERRD1... +0 SQLERRD2... +0
+00DA SQLERRD3... +0 SQLERRD4... FFFFFFFF SQLERRD5... +0
+00E6 SQLERRD6... +0 SQLWARN0... ' ' SQLWARN1... 'N'
+00EC SQLWARN2... ' ' SQLWARN3... ' ' SQLWARN4... ' '
+00EF SQLWARN5... '1' SQLWARN6... ' ' SQLWARN7... ' '
+00F2 SQLWARN8... ' ' SQLWARN9... ' ' SQLSTATE... '00000'
+00FC Statement number... +2896
+0106 Query command ID... 00000000 Query instance ID.... 00000000
+0118 QW0058ID... Scantype
+0118 Data type.... 'INDX' Rows processed.. +234 Rows examined.... +12
+012C Rows qualified... +7 After stage 1... +4 After stage 2.... +3
+0140 Rows inserted.... +17 Rows updated.... +12 Rows deleted..... +24
+0158 Pages scanned.... +76
+015C Pages scanned (RI)... +0 Rows deleted (RI).... +0
+0160 Pages scanned (LOB).. +0 Pages updated (LOB).. +0
+0188 QWHS....... Product section standard header
+0194 DB2 subsystem.... 'DBA1'
+01BC QWHSLWID... LUWID
+01BC Network ID... 'FTS1' LU name.... 'DBA1LU '
+01C4 Uniqueness value... CA80E6B51165 Commit count... +1
******************************** Bottom of data ********************************
Program statement
number 2896 caused an
index scan that
processed 234 rows in the
table

+002C QW0065..... IFCID data
Package
+002C Location... 'DB2BLOC' Collection ID.... 'MQATPGM'
+004E Package name... 'MQATPGM'
+0060 Consistency token.... 189E34F81745545D
Statement
+006A Statement type... 91 Cursor name.... 'C1' Isolation.... 'U'
+0080 Reoptimization... 0000 Statement number... +835
+0088 Cursor scrollability... 40
+0089 Cursor sensitivity... 40
+008A Result table type.... 40 Close commit... D5
+0094 Query command ID... 0
·———————————————————————————————— Field Zoom —————————————————————————————————·
| File Menu Help |
| ——————————————————————————————————————————————————————————————————————————— |
| BROWSE JCH.FUW.P0000003.D130625.T094351.EXTRACT + Line 00000000 |
| Command ===> Scroll ===> PAGE |
| ******************************* Top of data ******************************* |
| +007F QW0065I.... 'U' Isolation level of the SQL statement. |
| |
| Off QW0065RR... 'R' RR (repeatable read) |
| Off QW0065RS... 'T' RS (read stability) |
| Off QW0065CS... 'S' CS (cursor stability) |
| On QW0065UR... 'U' UR (uncommitted read) |
| Off QW0065XR... 'X' XR (Repeatable read with X lock) |
| Off QW0065XS... 'L' XS (Read stability with X lock) |
| ******************************* End of data ******************************* |
·—————————————————————————————————————————————————————————————————————————————·
Zoom
Zoom to see more detail about log record fields

——————————————————————————————————————————————————————————————————————————————
BROWSE JCH.FUW.P0000003.D130625.T094351.EXTRACT Record 00003251 More: < >
/ ———— Tracking ———————————————————————— Saturday 2013-06-22 Time (Elapsed)
380 SP entry FBOSPM3C DBA6 15.18.02.907449
TranCode=FBOIAP42 Userid=FUNTRM06 ClientID=ICDG
LUWID=FTS3/DBA6LU/CB8C9439E347/0001
————————————————————————————————————————————————————————————————————————————
380 SP exit FBOSPM3C SQLCODE=0000 DBA6 0.444391
TranCode=FBOIAP42 Userid=FUNTRM06 ClientID=ICDG
LUWID=FTS3/DBA6LU/CB8C9439E347/0001
————————————————————————————————————————————————————————————————————————————
003 Thread accounting DBA6 0.003521
TranCode=FBOIAP42 Program=FBOIAP42 Userid=FUNTRM06 Region=0001
RecToken=ICDG/0000000100000000 ClientID=ICDG
RESP=0.448242 CPU1=0.324230 CPU2=0.000791 I/O3=0.003360 Source=IMS_MPP
GtPgRq=284 SyPgUp=6 Suspnd=0 DeadLk=0 TimOut=0 MxPgLk=2
Sel=4 Ins=0 Upd=0 Del=1 LUWID=FTS3/DBA6LU/CB8C9439E347/0002
————————————————————————————————————————————————————————————————————————————
******************************* Bottom of Data ********************************
45
Life cycle events: expanded summary view
Scroll right to show the records in expanded view with elapsed or relative times:
Elapsed – time between log record events
Relative – time since start of transaction (or other selected event)

Identifying events for review or collaboration
Enter FIND LUWID on the command line.
Enter G to “tag” (bookmark) this DB2 record.
——————————————————————————————————————————————————————————————————————————————
BROWSE IMPOT01.SESSION7.TRANIX + Record 00005399 More: < >
/ ———— Tracking ———————————————————————— Saturday 2013-06-22 Time (Relative)
OrgUOWID=IADG/C62D2CB467860940 IMSID=IADG IMSRel=101
RecToken=IADG/0000003600000000
TotalTm=0.497791 RegTyp=MPP DBCalls=5
----------------------------------------------------------------------------
TAG IMS DB2 transaction with long response time +0.021122
----------------------------------------------------------------------------
G 0020 DB2 Unit of Recovery Control - Begin UR +0.021122
Userid=FUNTRM15 IMSID=IADG URID=00002A4010EA
LUWID=FTS3/DB3ALU/C62D2CB46A5A/0001
----------------------------------------------------------------------------
0020 DB2 Update In-Place in a Data Page +0.021138
DBID=0105 PSID=0002 URID=00002A4010EA
----------------------------------------------------------------------------
A DB2 expert can
now use the DB2
Log Analysis Tool
to investigate the
associated DB2
table updates,
based on the
transaction’s URID

Problem resolution: end of scenario
 The cause of the IMS transaction problem has been
narrowed down to a slowdown in DB2
 Sufficient information about the DB2 update activity has
been collected and can be passed on to the DB2 DBA for
further investigation
 Automatically locates log files for the problem time
range for supported subsystems
– SMF
– IMS log
– DB2 log and trace
 Enables a collaborative problem analysis:
– Between first responders and subject-matter experts
– Between experts in different areas47

48
SMF reports
 System events or constraints can affect transaction processing
 Workbench provides reports for selected SMF record types,
specifically aimed at identifying performance-related issues
Subsystem-related:
 SMF 33-2: APPC conversations
 SMF 88-1: System Logger
 SMF 101: DB2 accounting
 SMF 116: WebSphere MQ
System-related:
 SMF 30: Address Space activity;
including CICS, IMS, DB2
 RMF 70-1: CPU usage
 RMF 76: Page data sets
 RMF 78-2: Virtual Storage
 SMF 42-6: DASD data set I/O
 SMF 64: VSAM data set I/O

49
SMF 30: Address Space activity
----------Interval------------- System ---------- CPU ---------- EXCPs
Start Date/Time Duration Type Name Jobname Comp TCB SRB %CPU /Sec
2011-03-04 15:37:01 00:01 STE MVS1 IMSCTL1 0004 0.445357 0.023205 15.1 477
2011-03-04 15:37:01 00:01 STT MVS1 IMSCTL2 0004 0.445357 0.023205 15.0 476
2011-03-04 15:37:06 00:01 STE MVS1 IMSCTL3 0004 0.404175 0.011985 19.3 309
2011-03-04 15:37:06 00:01 STT MVS1 IMSCTL4 0004 0.404175 0.011985 19.2 309
2011-03-04 15:43:24 00:01 STE MVS1 IMSCTL5 0004 0.904357 0.046920 18.9 590
2011-03-04 15:43:24 00:01 STT MVS1 IMSVTL6 0004 0.904357 0.046920 18.9 589
2011-03-04 15:44:05 00:01 INT MVS2 CICSPR1 0000 7.966200 0.241357 15.2 140
2011-03-04 15:44:58 00:01 INT MVS2 CICSPR2 0000 0.141780 0.004335 11.2 100
---- System ---------- CPU ---------- EXCPs ----Storage----- -Paging/Sec-
Type Name Jobname Comp TCB SRB %CPU /Sec <16M >16M 64bit In Out Swap
STE MVS1 IMSCTL1 0004 0.445357 0.023205 15.1 477 1M 11M 0M 0 0 0
STT MVS1 IMSCTL2 0004 0.445357 0.023205 15.0 476 1M 11M 0M 0 0 0
STT MVS1 IMSCTL4 0004 0.404175 0.011985 19.2 309 1M 11M 0M 0 0 0
STT MVS1 IMSVTL6 0004 0.904357 0.046920 18.9 589 1M 11M 0M 0 0 0
INT MVS2 CICSPR1 0000 7.966200 0.241357 15.2 140 4M 1366M 0M 0 0 0
INT MVS2 CICSPR2 0000 0.141780 0.004335 11.2 100 0M 12M 0M 0 0 0
At regular intervals, every address
space can be monitored for unusual
spikes (or lulls) in system-related
resource consumption including CPU
and I/O.

50
Local Partner ----------- Time ---
Start Time LU Name Direction UserId Job Name SyncLvl InputQ Process
18:16:47.624543 MVSLU02 Outbound TWM#RBAT Syncpt .324737
** Partner ** TPname=IADGEXP_PROFILE
18:16:47.796620 IADGAPPC Inbound IADGMPPA Syncpt .166232 .154551
*** Local *** TPname=IADGEXP_PROFILE
SMF 33-2: APPC/MVS Conversation List report
Partner ----------- Time ----------- ------ Bytes ------
ection UserId Job Name SyncLvl InputQ Process Total Received Sent
bound TWM#RBAT Syncpt .324737 .324737 68 83
TPname=IADGEXP_PROFILE
ound IADGMPPA Syncpt .166232 .154551 .320783 83 68
TPname=IADGEXP_PROFILE
APPC requests processed on
z/OS are logged to SMF.
These requests may end up
being processed by an IMS or
CICS transaction.
A breakdown of processing
inside MVS can identify
bottlenecks and other
performance related issues.

SMF 42-6 DASD Data Set I/O report
 This example is for an IMS WADS (write-ahead data
set):
51
---------- DASD response time breakdown (average) ------------ ---- Maximum ----
Response Queuing Pending Connect Disc Rd Disc Wrt DAO Response Service
0.000512 0.000000 0.000000 0.000384 0.000000 0.000000 0.000000 0.001024 0.001024
ate rate per second ------ --- Cache I/O per sec ----
Read Hits Write Hits Seq RLC ILC
0 0% 0 0% 120 0 0
DSN: IADJ.VA10.WADS9
-- I/O per sec -- ---------- DASD response time breakdown (average) ----------
Reads Writes Response Queuing Pending Connect Disc Rd Disc Wrt D
0 120 0.000512 0.000000 0.000000 0.000384 0.000000 0.000000 0.0000
---- Cache candidate rate per second ------ --- Cache I/O per sec ----
Total Hits Read Hits Write Hits Seq RLC ILC
0 0% 0 0% 0 0% 120 0 0

52
--Splits-- -------Calls------- -RLS Activity-
CA CI Ext EXCPs Get Upd Del Ins LSR CF DASD
VS.RKM2EDS3.DATA 5 17 1 3322 13 1 0 1314 0 0 0
VS.RKM2EDS3.INDEX 0 5 1 1796 0 259 0 0 0 0 0
VS.RKM2EDS3.DATA 5 18 1 3378 13 1 0 1340 0 0 0
VS.RKM2EDS3.INDEX 0 5 1 1850 0 261 0 0 0 0 0
VS.RKM2EDS3.DATA 5 19 1 3436 13 1 0 1353 0 0 0
VS.RKM2EDS3.INDEX 0 5 1 1902 0 275 0 0 0 0 0
DATA 0 0 1 3754685 23K 8658 4353 1602 0 0 0
INDEX 0 0 1 3739616 13 0 0 0 0 0 0
--Splits--
Close Date Time Data set name CA CI Ext EXCPs
2011-02-02 16:00:01 FUNDIP.OME.FTS1MVS.RKM2EDS3.DATA 5 17 1 3322
2011-02-02 16:00:01 FUNDIP.OME.FTS1MVS.RKM2EDS3.INDEX 0 5 1 1796
2011-02-02 16:00:55 FUNDIP.ANF.QUEUE.DATA 0 0 1 3754685
2011-02-02 16:00:55 FUNDIP.ANF.QUEUE.INDEX 0 0 1 3739616
SMF 64: VSAM Data Sets
VSAM data sets are commonly
used as databases in IMS and
CICS.
As these data sets are re-opened
(or extend), information about their
I/O activity and general health
(splits) is available.

53
----------- Number of Address Spaces -----------
---- In --- -In Ready- -Out Ready- -Out Wait-
Avg Max Avg Max Avg Max Avg Max
151 156 7 86 0 1 0 0
77 80 1 15 0 0 0 0
69 72 1 9 0 0 0 0
------------------------ Number
- Interval Start -- System - %CPU Busy - IO ---- In --- -In Ready- -Out R
Date Time Name LPAR MVS Rate Avg Max Avg Max Avg
2010-08-17 23:45:00 FTS1 68.75 87.42 2282.4 151 156 7 86 0
FTS2 4.07 4.50 9.4 77 80 1 15 0
FTS3 4.03 4.39 12.6 69 72 1 9 0
2010-08-18 00:00:00 FTS1 61.15 72.16 1934.8 150 155 5 76 0
FTS2 4.15 4.72 8.4 77 79 1 8 0
FTS3 3.88 4.41 11.7 69 73 1 13 0
SMF 70-1: RMF Processor activity
CPU constraints are
one of the most
common causes of a
slowdown in
performance, and often
has flow-on effects
including contention.
Out Ready identifies the number of address
spaces waiting for dispatching on the CPU
CPU Busy and IO Rate
are the classical system
performance indicators.
Look for spikes that might
indicate a slowdown.

54
SMF 75: RMF Page Data Set activity
Date: 2010-08-17 Time: 23:45:00 SID: FTS1
Page -------- Slots Used ------- % Bad In Trans Number Pages
Type Alloc Min Max Avg Full Slots Use Time I/O Req Xferd VIO Dat
PLPA 44999 20078 20078 20078 45% 0 0 0 0 0 FUN
Common 89999 3129 3129 3129 3% 0 0 0 0 0 FUN
Local 1080K 101K 101K 101K 9% 0 0 0 10 10 Y FUN
Local 1080K 102K 102K 102K 9% 0 0 0 10 10 Y FUN
Local 1080K 103K 103K 103K 10% 0 0 0 6 6 Y FUN
Local 1080K 109K 109K 109K 10% 0 0 0 13 13 Y FUN
Time: 23:45:00 SID: FTS1
lots Used ------- % Bad In Trans Number Pages
Min Max Avg Full Slots Use Time I/O Req Xferd VIO Data Set Name
078 20078 20078 45% 0 0 0 0 0 FUNDI1.FTS1.PAGE.PLPA
129 3129 3129 3% 0 0 0 0 0 FUNDI1.FTS1.PAGE.COMMON
01K 101K 101K 9% 0 0 0 10 10 Y FUNDI1.FTS1.PAGE.LOCAL1
With the advent of large amounts of cheaper memory,
Page data set performance is often less of a problem
today, but none the less should be monitored
occasionally for constraints.

55
val Start -- System ---------------------- Usage -----------------------
Time Name Type Size Min Time Max Time Avg Pct
-13 23:45:00 FTS1 CSA 3364K 612K 23:44:59.60 612K 23:44:59.60 612K 18.2
ECSA 384M 131M 23:44:59.60 131M 23:48:49.24 131M 34.1
SQA 1744K 444K 23:44:59.60 444K 23:44:59.60 444K 25.5
ESQA 47772K 22120K 23:47:19.06 22212K 23:44:59.60 22177K 46.4
FTS2 CSA 3364K 376K 23:44:59.60 376K 23:44:59.60 376K 11.2
SMF 78-2: RMF Virtual Storage activity
- Interval Start -- System ---------------------- Usage --------------
Date Time Name Type Size Min Time Max Time
2010-06-13 23:45:00 FTS1 CSA 3364K 612K 23:44:59.60 612K 23:44:59.60 6
ECSA 384M 131M 23:44:59.60 131M 23:48:49.24 1
SQA 1744K 444K 23:44:59.60 444K 23:44:59.60 4
ESQA 47772K 22120K 23:47:19.06 22212K 23:44:59.60 221
FTS2 CSA 3364K 376K 23:44:59.60 376K 23:44:59.60 3
IMS and CICS still use large amounts of CSA and ECSA for
common storage. In the event that storage cannot be obtained,
subsystems can stop or worse.

56
Reg Max CICS
ancode PSBname PST Typ Duration Locks Recovery Token Resource Task
1CSAC3 PCM0F0 49 11.534336 0 CI1CSAC3/C5BF632F08B62783 HNMTRM01 00088603
1ESAE1 PCM0F0 127 111.149056 44 CI1ESAE1/C5BF62D0456F8085 00036462
1ESAE5 PCM0F0 102 11.534336 0 CI1ESAE5/C5BF63D077B36503 HNMTRM01 00088040
1FSAF3 PCM0F0 40 98.566144 44 CI1FSAF3/C5BF637DEF1A2001 00032398
1ASAA2 PRE0F0 90 11.534336 26 CI1ASAA2/C5BFBB316C472003 SHSECN08 00013029
1ASAA1 PSA0F0 60 11.534336 2 CI1ASAA1/C5BFBB3166E1F584 00048273
SMF 79-15: IRLM Long Lock Detection
Cycle Entry Reg Max
Time Number Type IMS ID Trancode PSBname PST Typ Duration Locks Reco
08:51:47.440 25853771 Wait ISA2 CI1CSAC3 PCM0F0 49 11.534336 0 CI1C
08:51:47.440 25853771 Block ISA3 CI1ESAE1 PCM0F0 127 111.149056 44 CI1E
08:54:36.250 25854107 Wait ISA3 CI1ESAE5 PCM0F0 102 11.534336 0 CI1E
08:54:36.250 25854107 Block ISA4 CI1FSAF3 PCM0F0 40 98.566144 44 CI1F
15:25:31.580 25900783 Wait ISA1 CI1ASAA2 PRE0F0 90 11.534336 26 CI1A
15:25:31.580 25900783 Block ISA1 CI1ASAA1 PSA0F0 60 11.534336 2 CI1A
IMS database locks that are held
by transactions for an extended
period (several seconds) are
logged to SMF; and can be
analyzed to determine if there is
an application problem.

57
SMF 88-1: System Logger Log Stream
Logstream name MVSID Structure name Group First interval start
STC@CICS.CICSPR1.DFHLOG FTS1 *DASDONLY* 14:30:00.00 3/04/2011
----------- IXGWRITES ---------- ---------------- DELETIONS ---
Bytes Count Count Byt
Writn to With Without Aft
Total Average Interim DASD DASD Offlo
Count Bytes Bytes Storage Write Write w. DA
-------- -------- -------- -------- -------- -------- ------
Total 29862 19177K 642 122692K 24950 4075 1025
Rate(/Sec) 0 5 35 0 0
Minimum 0 0 0 0 0
Maximum 1322 862741 5480448 1685 839 70328
tructure name Group First interval start Last interval stop Total Interval
DASDONLY* 14:30:00.00 3/04/2011 16:00:00.00 4/14/2011 0985:30:00
------- ---------------- DELETIONS -----------------
Bytes Count Count Bytes Bytes
Writn to With Without After Int Stor
Average Interim DASD DASD Offload w/o DASD
Bytes Storage Write Write w. DASD Write
------- -------- -------- -------- -------- --------
642 122692K 24950 4075 102547K 16691K
35 0 0 29 5
0 0 0 0 0
5480448 1685 839 7032832 3436544
CICS and IMS both rely
on log streams for
critical services;
including message
handling and journaling.
Problems can be
avoided by monitoring
their I/O and offload
activity.

58
SMF 101: DB2 Thread Accounting
DB2 Plan ---------- Connection ----------- Thread
SSID Name Name Type Count
DB3A CEXTPGM IADG IMS MPP 68
Start: 2010-06-24 15:27:39
End: 2010-06-24 16:44:00
Class1: Thread Time Avg: Elapsed=70.43305 CPU= .011006 Interval: 01:16:20
Max: Elapsed=2045.732 CPU= .013724 Rate/sec: < 1
Class2: In-DB2 Time Avg: Elapsed= .015108 CPU= .006035
Max: Elapsed= .033537 CPU= .008234
Class3: Suspend Time Avg: Total = .008709 I/O= .000000 Lock/Latch= .002404 Other= .006305
Max: Total = .017377 I/O= .000000 Lock/Latch= .007199 Other= .010178
Buffer Manager Summary Avg: GtPgRq= 7.0 SyPgUp= 3.0
Max: GtPgRq= 7 SyPgUp= 3
Locking Summary Avg: Suspnd= .0 DeadLk= .0 TmeOut= .0 MxPgLk= 1.0
Max: Suspnd= 0 DeadLk= 0 TmeOut= 0 MxPgLk= 1
SQL DML Query/Update Avg: Sel= .0 Ins= 1.0 Upd= 1.0 Del= 1.0
Max: Sel= 0 Ins= 1 Upd= 1 Del= 1
SQL DML 'Other' Avg: Des= .0 Pre= .0 Ope= 1.0 Fet= 9.0 Clo= 1.0
Max: Des= 0 Pre= 0 Ope= 1 Fet= 9 Clo= 1
All transactions that use
DB2 cut accounting records
that show how DB2
performed in the application
and across into DB2.

59
SMF 116: WebSphere MQ Accounting reports
MQACCT4 Printed at 10:50:30 2/03/2011 Data from 09:00:40 03/03/2010 to 09:59:52 03/03/2010
SSID: SYSB Type: CICS Name: CICSSYSP Tran: TRTI Threads: 2
Other Avg Count 6.0 Avg Elapsed 0.000116 Avg CPU 0.000112
In-MQ Time (Total) Elapsed: 0.000233 CPU: 0.000224
In-MQ Time (Average) Elapsed: 0.000116 CPU: 0.000112
SSID: SYSB Type: CICS Name: CICSSYSP Tran: TRTL Threads: 4
In-MQ Time (Total) Elapsed: 0 CPU: 0
In-MQ Time (Average) Elapsed: 0 CPU: 0
Queue: APPLICATION_A_REQUEST
QType: LOCAL IType: NONE GDisp: Q_MGR QCount: 4
Count Elapsed CPU Susp Elp JnlWrt Elp PS Req's PS Rd Elp Expir
_________ _________ _________ _________ __________ _________ _________ ______
OPEN 15.0 0.000019 0.000009
CLOSE 15.0 0.000002 0.000002
INQ 15.0 0.000009 0.000008
In-MQ Time (Total) Elapsed: 0.001861 CPU: 0.001222
In-MQ Time (Average) Elapsed: 0.000465 CPU: 0.000305
Detailed MQ accounting
can be requested to show
the impact of MQ on
transaction performance.

60
File Edit Edit_Settings Menu Utilities Compilers Test Help
———————————————————————————————————————————————————————————————————————————————
VIEW FUW110.WTWM.REPORTS(OPERLOG) - 01.03 Columns 00001 00072
****** ***************************** Top of Data ******************************
000001 //OPERLOG JOB ,CLASS=A,NOTIFY=&SYSUID
000002 /*JOBPARM SYSAFF=FTS1
000003 //FUWBATCH EXEC PGM=FUWBATCH
000004 //STEPLIB DD DISP=SHR,DSN=FUW.SFUWLINK
000005 //SYSPRINT DD SYSOUT=*
000006 //SYSIN DD *
000007 LOGSTREAM OPERLOG:SYSPLEX.OPERLOG
000008 START 2011-04-06-08.40.00.00 STOP 2011-04-06-09.00.00.00
000009 REPORT OPERLOG
000010 CODE(OPERLOG)
000011 COND TEXT(2) EQ 'DFS'
000012 COND TEXT(*) EQ 'BACKOUT'
000013 /*
****** **************************** Bottom of Data ****************************
OPERLOG report: JCL
Note the LOGSTREAM command in the SYSIN data set, identifying the input
log stream. The COND statements filter the reported OPERLOG records.

61
OPERLOG report: output
FTS3 2011096 08.41.42.57 STC36951 DFS2484I JOBNAME=IBB1#ARC
GENERATED BY LOG AUTOMATIC ARCHIVING IBB1
FTS2 2011096 08.41.48.71 STC37128 DFS058I 08:41:48 START COMMAND IN PROGRESS ICDZ
FTS2 2011096 08.41.49.80 STC37128 DFS551I IFP REGION ICDZIFP1 STARTED
ID=00001 TIME=0841 ICDZ
FTS2 2011096 08.41.49.89 STC37128 DFS551I MESSAGE REGION ICDZMPP1 STARTED
ID=00002 TIME=0841 CLASS=001,000,000,000 ICDZ
FTS2 2011096 08.41.52.04 STC37128 DFS551I IFP REGION ICDZIFP3 STARTED
ID=00003 TIME=0841 ICDZ
FTS3 2011096 08.47.36.05 STC36951 DFS554A FUWTCIC 00002 FUWTCIC DFHTWM04(3)
000,0777 2011/096 8:47:36
RTKN=FUWTCIC C79459EA853EFB03 IBB1
FTS3 2011096 08.47.51.05 STC36951 DFS968I DBD=DI21PART WITHIN PSB=DFHTWM04
SUCCESSFULLY BACKED OUT IBB1
FTS3 2011096 08.47.51.05 STC36951 DFS980I BACKOUT PROCESSING HAS ENDED FOR DFHTWM04 IBB1
From the previous JCL request, it is simple to identify the IMS subsystem
messages associated with the transaction failure.

CICS DBCTL
 For all CICS transaction you can analyze CMF (SMF 110) records.
 However CMF records do not contain all of the details of the IMS
events that are triggered by DL/I requests. Those details are in the
IMS log.
 Transaction Analysis Workbench enables you to analyze CICS
DBCTL transactions using:
– CMF records
– IMS log records
– A combined view of both
 In particular Workbench enables you to analyze exceptions:
CICS DBCTL transactions that abended or had a long response
time.
62

Example CICS-DBCTL summary report
63
ss
CICS
ss
IMS
2014-03-26 Wednesday CICS-DBCTL Summary Page 1
Tran APPLID CMF Count Response CPU Time IMS Reqs IMS Wait ABEND Rate/Sec
---- -------- --------- -------- -------- -------- -------- -------- --------
BANK CICSP1 60 11.12982 0.008967 35 4.256977 10 0
08 Count Elapsed CPU Time StaDelay Schedule IC Wait PS Wait
--------- -------- -------- -------- -------- -------- --------
42 10.94999 0.004092 0.011668 0.000183 0 0
07 Count DB call DB Gets DB Upds IO Count IO Time LockWait
--------- -------- -------- -------- -------- -------- --------
41 33 13 19 4 0.003438 3.980170
FP Count FP call FP Gets FP Upds FP Wait FP Fail
--------- -------- -------- -------- -------- --------
41 19 7 11 0 7
Synctime Phase 1 Phase 2 FP PH2 OTHREAD
-------- -------- -------- -------- --------
0.011938 0.006555 0.005383 0.002232 0.017659

Ims12 workbench data visualization - IMS UG May 2014 Sydney & Melbourne

More Related Content

Ims12 workbench data visualization - IMS UG May 2014 Sydney & Melbourne