DFSort Tuning Guide
DFSort Tuning Guide
DFSort Tuning Guide
Version 2 Release 3
IBM
SC23-6882-30
Note
Before using this information and the product it supports, read the information in “Notices” on page
85.
This edition applies to Version 2 Release 3 of z/OS (5650-ZOS) and to all subsequent releases and modifications until
otherwise indicated in new editions.
Last updated: 2019-02-16
© Copyright International Business Machines Corporation 1992, 2017.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with
IBM Corp.
Contents
iii
Central storage..................................................................................................................................... 14
Storage control cache.......................................................................................................................... 14
Disk....................................................................................................................................................... 14
Tape...................................................................................................................................................... 15
Virtual storage ........................................................................................................................................... 15
Main storage......................................................................................................................................... 16
System-managed storage..........................................................................................................................16
iv
COBOL Calling Program for Method 1..................................................................................................54
Operation (NOFASTSRT in effect)........................................................................................................ 55
Performance......................................................................................................................................... 56
Method 2: COBOL program with DFSORT control statements ................................................................ 56
Operation (FASTSRT in effect)............................................................................................................. 56
Productivity...........................................................................................................................................57
Control statements...............................................................................................................................57
COBOL calling program........................................................................................................................ 57
Performance......................................................................................................................................... 58
Method 3: DFSORT with control statements ............................................................................................59
Control statements...............................................................................................................................59
Operation.............................................................................................................................................. 59
Productivity...........................................................................................................................................59
Performance......................................................................................................................................... 59
Appendix B. Accessibility.....................................................................................81
Accessibility features.................................................................................................................................81
Consult assistive technologies.................................................................................................................. 81
Keyboard navigation of the user interface................................................................................................ 81
Dotted decimal syntax diagrams............................................................................................................... 81
Notices................................................................................................................85
Terms and conditions for product documentation................................................................................... 86
IBM Online Privacy Statement.................................................................................................................. 87
Policy for unsupported hardware.............................................................................................................. 87
Minimum supported hardware.................................................................................................................. 88
Programming interface information.......................................................................................................... 88
Trademarks................................................................................................................................................ 88
Index.................................................................................................................. 89
v
vi
List of Figures
vii
viii
List of Tables
ix
x
About this document
Sorting is one of the most frequently used functions at most data processing sites.
This document provides information about tuning IBM DFSORT, offering suggestions for reducing its use
of system resources and achieving better turnaround time for the many applications that use DFSORT
without adversely affecting system performance.
This document is intended for systems engineers, performance analysts, system programmers, and
application programmers, who have some experience with DFSORT. For those unfamiliar with DFSORT,
z/OS DFSORT Installation and Customization and z/OS DFSORT Application Programming Guide provide
information that will help you use this document effectively.
In this document, one gigabyte (GB) is equal to 1024 megabytes (MB), which is equal to 1048576
kilobytes (KB), which is equal to 1073741824 bytes.
Referenced documents
This document refers to the following documents:
z/OS DFSORT Tuning Guide is a part of a more extensive DFSORT library. These documents can help you
work with DFSORT more effectively.
Task Documentation
Planning for and customizing DFSORT z/OS DFSORT Installation and Customization
Learning about DFSORT z/OS DFSORT: Getting Started
Application programming z/OS DFSORT Application Programming Guide
Interpreting messages and codes, and z/OS DFSORT Messages, Codes and Diagnosis Guide
diagnosing failures
z/OS information
This information explains how z/OS references information in other documents and on the web.
When possible, this information uses cross document links that go directly to the topic in reference using
shortened versions of the document title. For complete titles and order numbers of the documents for all
products that are part of z/OS, see z/OS Information Roadmap.
To find the complete z/OS® library, go to IBM Knowledge Center (www.ibm.com/support/
knowledgecenter/SSLTBW/welcome).
Chapter 1. Introduction
This document offers information and recommendations on tuning DFSORT. It provides advice to the
system programmer on setting DFSORT's installation default parameters as well as to the application
programmer on improving the performance of individual DFSORT applications. It describes the main
indicators used to measure performance and emphasizes the advantages of using DFSORT's Blockset
technique for improved performance. Since sort applications tend to be more complex and time-
consuming than merge or copy applications, this document concentrates on techniques for improving the
performance of sort applications. However, many of these techniques are also appropriate for copy and
merge applications.
This document also assumes that DFSORT's most efficient technique is used. See “Blockset technique”
on page 5 for more information on ensuring that Blockset is used.
Note: Although a DFSORT “application” can comprise one or more parts of a job, for simplicity's sake the
terms “application” and “job” are used interchangeably in this document.
This chapter describes the following:
• The importance of tuning DFSORT
• Examples of successful tuning of DFSORT
• System resources and tuning DFSORT
• The performance indicators for DFSORT
ftp://ftp.software.ibm.com/storage/dfsort/mvs/ (ftp://ftp.software.ibm.com/storage/dfsort/mvs/)
Successful tuning
Some of the practical advice in this document is based on the experience of IBM developers working with
DFSORT customers to help them in specific situations.
This document offers many recommendations, ranging from advice on setting the most appropriate option
values to guidelines for optimizing virtual storage and work space. You should decide which
recommendations are best for your site, based on site requirements and the amount of resources
available to implement them.
System resources
The purpose of tuning DFSORT is to use system resources more efficiently. This is important at most sites,
since there are usually too many demands made for limited resources. Although DFSORT automatically
optimizes many of its tuning decisions, there are additional actions which a site and its DFSORT users can
take to further improve DFSORT performance. These actions depend on the priority given to various
performance objectives.
Different sites and programmers define efficient performance in different ways. A site with a primarily
batch environment or an application programmer with a single task to complete probably measures
performance based on elapsed time. Alternatively, a site experiencing high CPU usage or a programmer
who is charged based on CPU time is more likely to evaluate efficiency in terms of reduced CPU time.
While improved performance is the objective of tuning, often it is necessary to compromise. That is,
improving the use of one system resource can have a negative impact on other resources, in much the
same way that giving more resources to one application can make it perform much better at the expense
of degrading the performance of other applications that are competing for the same resources.
The main trade-offs that you should consider are among:
Processor Load
Accounting charges are often based on the number of CPU service units used by a job or address
space. The more CPU time used, the higher the charges. Reducing a job's CPU time not only reduces
these charges, but also enables other jobs competing for the same CPU resource to complete sooner.
Paging Activity
System paging activity reflects how the system is managing central storage to meet the virtual storage
requirements of applications and user programs. Paging is the process of moving virtual storage pages
from central storage to auxiliary storage, and takes a large amount of elapsed time to perform.
System paging activity can increase elapsed time for user programs. If the activity is too high, the
system reduces its workload by reducing the number of jobs running, and might suspend processing
of some jobs (known as swapping) until the paging activity returns to an acceptable level. The result is
that the system spends more time managing virtual storage, while many user programs take longer to
complete.
I/O Activity
Some accounting charges are based on the I/O performed by a job. This is frequently measured by
execute channel program (EXCP) counts. While EXCP counts might not represent a completely
accurate usage of I/O resources, they are important to many users and sites, and steps can be taken
to reduce them.
Elapsed Time
Elapsed time is likely to be most important for sites with a limited batch window, where processing of
particular applications has to be completed within a certain period. It is also important to users whose
productivity depends on having their applications complete as soon as possible.
Disk Utilization
In environments where disk space is constrained, the amount of auxiliary storage required by an
application is a primary concern. For DFSORT applications, this usually involves the efficient use of
output and work data sets.
Performance indicators
This section describes how you can measure the performance factors listed previously for DFSORT. Often,
performance is evaluated as a combination of some or all of them. The priorities given to each depend
upon site objectives.
Processor utilization
CPU fields are used to measure the amount of work performed by the processor, as opposed to work
performed by the I/O and storage subsystems. CPU time consists of the sum of the following five
components.
Task Control Block (TCB)
The CPU time used to perform user program activity on behalf of user tasks for a job step.
Service Request Block (SRB)
The CPU time used to perform system service requests on behalf of user tasks for a job step.
Hiperspace™ Processing Time (HPT)
If the user task reads from or writes to a Hiperspace using the HSPSERV macro, this is the amount of
CPU time used to service these reads and writes.
I/O Interrupt Processing (IIP)
The CPU time used to handle input/output (I/O) interrupts on behalf of user tasks for a job step.
Region Control Task (RCT)
If the user task is swapped out while running, this is the amount of CPU time used to swap the task
out and back in again.
Introduction 3
Introduction
I/O activity
This represents the movement of data between the processor and disk or tape devices. The effective use
of I/O resources is important to a site and I/O activity is often an important component of site accounting
methodologies.
Because performing input to and output from disk and tape devices typically takes much longer than
manipulating the data in real storage, the amount of I/O performed is a key component of an application's
Elapsed time. Therefore, reducing I/O generally improves an application's Elapsed time.
I/O activity is primarily measured in the following ways:
EXCPs
The number of execute channel program (EXCP) commands issued (logical I/Os)
SSCHs
The number of start subchannel (SSCH) commands issued (physical I/Os)
Device Connect Time
The amount of time a particular device is dedicated for the I/O transfer
Channel Usage
The percentage of time a channel is busy initiating, transferring, or completing the movement of data
between a device and the CPU
While EXCPs are often used as a measure of I/O activity, their counts can be extremely misleading. For
example, one EXCP can be used to transfer a few bytes or dozens of megabytes! SSCHs measure the
number of physical I/Os to a data set and thus can be more useful than EXCPs. A complete analysis of
total I/O performance should consider device connect time, channel usage and SSCHs.
Elapsed time
Elapsed time refers to the amount of “wall clock” time from initiation to termination of the application. For
typical sorting applications, elapsed time is composed primarily of I/O time, with CPU time and I/O
queueing time also contributing significantly.
The Elapsed time for an application can differ from run to run, depending upon the amount of competition
from other applications for system resources. Accurate Elapsed time comparisons can be done only if the
system has no other applications running.
Disk utilization
In certain environments, disk space usage is a more important characteristic of an application's
performance than CPU time or elapsed time. Inefficient disk usage is usually measured in terms of the
amount of disk space that is allocated, compared to the amount of disk space actually needed.
Frequently, large amounts of disk space are wasted as a result of inefficient blocking
The performance of a DFSORT application is largely determined by the use of a special set of product
features. This chapter provides an introduction to the performance features of DFSORT including:
• Blockset technique
• OUTFIL
• Memory object sorting, Hipersorting and dataspace sorting
• Dynamic Storage Adjustment
• Cache fast write
• ICEGENER
• Compression
• Striping
• Dynamic allocation of work data sets
• System-determined block size
• Larger tape blocksizes (greater than 32K)
• Managed tape data sets
• IDCAMS BLDINDEX
• DFSORT's Performance Booster for The SAS System
How to use these features to gain the most effective performance from DFSORT is described in Chapter 4,
“Installation considerations,” on page 19 and in Chapter 5, “Run-time considerations,” on page 35.
Blockset technique
The Blockset technique is DFSORT's most efficient method for sorting, merging, and copying. Blockset
uses optimized algorithms and makes the most efficient use of IBM hardware. DFSORT uses Blockset
whenever possible. DFSORT's other techniques, Peerage/Vale and Conventional, are not as efficient as
Blockset and are only used when Blockset cannot be used.
The Blockset technique can reduce CPU time, I/O activity, and Elapsed time. It is strongly recommended
that you remove any obstacles to using Blockset whenever possible. For the purposes of this document,
any recommendations to improve performance assume that Blockset is the DFSORT technique used.
It is worth checking to see if Blockset was used for a given application; message ICE143I in the SYSOUT
data set shows which technique was used. If ICE143I is not shown or shows that a technique other than
Blockset was selected, resubmit the application with a SORTDIAG DD statement (unnecessary if the
application already had a SORTDIAG DD statement or if your site has specified installation option
DIAGSIM=YES). Additional DFSORT messages and diagnostic information are then shown, including:
• ICE802I, which shows the technique used
• ICE800I or error messages
ICE800I gives a code indicating why Blockset was not used. If the code is 1, one or more error messages
are also shown.
Note: Blockset messages are suppressed if another technique is selected, unless a SORTDIAG DD
statement is present, or installation option DIAGSIM=YES has been specified for your site.
Blockset will not be selected if you specify work data sets on tape devices. Blockset only supports tape
devices for input and output data sets. It is very strongly recommended that you use disk rather than tape
for work data sets.
OUTFIL
OUTFIL is a control statement that allows you to create one or more output data sets for a sort, copy, or
merge application with only one pass over an input data set. You can use multiple OUTFIL statements,
with each statement specifying the OUTFIL processing to be performed for one or more output data sets.
With a single pass over an input data set, OUTFIL can create one or more:
• Output data sets containing unedited or edited records
• Output data sets containing different ranges, samples, or subsets of records. Those records that are not
selected for any subset can be saved in another output data set.
• Reports containing different header records, detail records and trailer records.
• FB output data sets from a VB input data set.
• VB output data sets from an FB input data set.
OUTFIL has many additional features as well; see z/OS DFSORT Application Programming Guide for
complete details on all of the tasks you can perform with OUTFIL.
Benefits
Because OUTFIL allows you to create output data sets with only one pass over an input data set, OUTFIL
can improve performance. This is especially true for elapsed time and EXCPs when you compare an
OUTFIL job to create many output data sets with many non-OUTFIL jobs each creating one output data
set.
Benefits
Operation
In addition to DFSORT, central storage can be used by many other applications and system components,
such as DB2® buffer pools, VSAM Hiperspace, Hiperbatch, virtual lookaside facility (VLF), VIO, and the
paging subsystem. To avoid overusing storage, which can lead to paging data set space shortages and
increased system paging activity, DFSORT uses several safeguards.
To begin with, DFSORT is always aware of the future storage needs of other concurrent DFSORT
applications. A DFSORT application never attempts to back its memory object, Hiperspace or data space
data with storage that is needed by another DFSORT application.
Next, DFSORT dynamically determines the amount of available storage prior to creating each memory
object, Hiperspace or data space. "Available" storage is the storage used to back new Memory Object,
Hiperspace or data space data. It consists of two types:
• Free storage is storage that is not being used by any application.
• Old storage is storage that is being used by other applications, but whose data has been unreferenced
for a long period of time. This time period is long enough that the system deems it eligible for migration
to auxiliary storage to make room for new data.
Knowing both the amount of storage needed by other DFSORT applications and the amount of available
storage, along with the values of DFSORT installation options EXPMAX, EXPOLD, EXPRES and TUNE,
DFSORT can assess the impact of creating a memory object, Hiperspace or data space. This greatly
reduces the likelihood of overusing central storage, especially as a result of multiple large concurrent
DFSORT applications. It also allows sites to customize their total use of storage by DFSORT applications
through the EXPMAX, EXPOLD, EXPRES and TUNE installation options. These can only be specified as
installation wide defaults using an ICEPRMxx member of PARMLIB or the ICEMAC macro.
MOSIZE, HIPRMAX and DSPSIZE control the amount of memory object, Hiperspace and data space each
individual DFSORT application can use while EXPMAX, EXPOLD, and EXPRES control the total amount of
storage used by all DFSORT applications running in the system. These can be specified as installation
wide defaults using an ICEPRMxx member of PARMLIB or the ICEMAC macro, overridden at run-time with
an EXEC PARM option or OPTION control statement, or overridden with an installation wide ICEIEXIT exit
routine.
The MOWRK installation default controls whether memory objects can be used as intermediate work
space. If MOWRK=NO is in effect, then memory objects can only be used as additional main storage.
MOWRK can be specified as an installation wide default using an ICEPRMxx member of PARMLIB or the
ICEMAC macro, or overridden at run-time with an EXEC PARM option or OPTION control statement.
MOWRK cannot be overridden with an ICEIEXIT exit routine.
When memory objects or data spaces are used as additional main storage, each sort creates one large
area at the beginning and no adjustments are made until the memory object or data space is deleted
when the sort terminates.
When memory objects or Hiperspaces are used as intermediate work space, DFSORT has the ability to
create up to 16 memory objects or Hiperspaces which can be allocated incrementally or all at once. The
advantage of allocating storage in increments is that multiple concurrent sorts can more evenly share
available central storage resources since available central storage is evaluated prior to creating each new
memory object or Hiperspace. When storage is allocated in this way, DFSORT's dynamic allocation of work
data sets must allocate additional space to safeguard against any unexpected increase in disk work space
requirements if central storage resources become constrained during the sort. When each sort allocates
all of it's memory object or Hiperspace immediately, DFSORT's dynamic allocation can be more
aggressive in reducing space allocations since it is less likely that the disk work space allocations will be
larger than expected. However, when multiple sorts are running concurrently, a small number of them
may monopolize the available central storage while others are forced to use only disk work space.
DFSORT provides the TUNE installation default to control how DFSORT evaluates available resources and
allocates memory objects, Hiperspaces and data spaces. If balancing storage usage by multiple
concurrent sorts is a priority, the shipped default of TUNE=STOR is recommended. If conserving disk work
space is a priority, TUNE=DISK is recommended. TUNE can be specified as an installation wide default
using an ICEPRMxx member of PARMLIB or the ICEMAC macro. TUNE cannot be overridden with an
ICEIEXIT exit routine.
The following are actions you can take which might increase DFSORT's use of memory objects,
Hiperspaces and data spaces:
• Ensure that MOSIZE=MAX, MOWRK=YES, HIPRMAX=OPTIMAL and DSPSIZE=MAX are used. These
parameters can be specified as installation-wide defaults using an ICEPRMxx member of PARMLIB or
the ICEMAC macro, or overridden at run-time with an EXEC PARM option or OPTION control statement.
MOSIZE, HIPRMAX and DSPSIZE can also be overridden with an installation-wide ICEIEXIT exit routine.
• Verify that a sufficient size for memory objects is defined by the MEMLIMIT parameter on the JOB or
EXEC JCL statement. Note that if REGION=0M is specified, MEMLIMIT=NOLIMIT is assumed. If REGION
is set equal to a value other than 0M and MEMLIMIT is not specified, the default MEMLIMIT value
defined in the SMFPRMxx member of PARMLIB is used.
• Verify that IEFUSI does not place any restrictions on the size of the memory objects, Hiperspaces or
data spaces a single address space can create.
• Ensure that DFSORT has accurate information about the input file size. DFSORT can automatically
estimate the file size for disk input data sets, and tape data sets managed by DFSMSrmm or a tape
management system that uses ICETPEX. See “File Size and Dynamic Allocation” in z/OS DFSORT
Application Programming Guide for information on situations where DFSORT cannot determine the file
size accurately, and what to do about it.
• Verify that there is sufficient available central storage present to back DFSORT's memory objects,
Hiperspaces and data spaces.
See Chapter 4, “Installation considerations,” on page 19 and Chapter 5, “Run-time considerations,” on
page 35 for further details on how best to tune DFSORT's use of memory objects, Hiperspaces and data
spaces.
Benefits
Increasing the amount of virtual storage available to DFSORT can significantly improve the performance of
many sort applications, especially large sorts. The optimal amount of virtual storage varies with the
amount of data being sorted. Consequently, the best performance and most efficient use of virtual storage
can only be obtained by tuning the virtual storage of each individual sort. DSA does most of this tuning
automatically, relieving system and application programmers of the task. By using the optimal amount of
virtual storage, DFSORT maximizes its performance while minimizing the impact of DFSORT applications
on the system.
Operation
DFSORT's use of virtual storage from the primary address space is limited by several factors, most notably
system (for example, REGION, IEFUSI) and DFSORT (for example, TMAXLIM, SIZE) parameters, defaults,
and exits. While DFSORT can never exceed the system limits, those limits usually exceed the virtual
storage DFSORT needs for most applications. As a result, the DFSORT limits usually control the amount of
virtual storage used by DFSORT.
Dynamic Storage Adjustment permits a range to be specified for the default amount of virtual storage for
sorts, provided SIZE/MAINSIZE=MAX is in effect. This range starts with the value specified for TMAXLIM
and goes up to the value specified for DSA. For most sorts, DFSORT limits its use of virtual storage to
TMAXLIM. However, for larger sorts, DFSORT automatically optimizes performance by increasing the
amount of virtual storage it uses, up to but not exceeding the DSA value.
See Chapter 4, “Installation considerations,” on page 19 and Chapter 5, “Run-time considerations,” on
page 35 for further details on the best way to tune DFSORT's use of DSA.
Benefits
You can gain elapsed time savings by using cache fast write. The benefits of CFW are twofold:
• When writing data to the work data sets, the write operations can complete at cache speed rather than
at disk speed, as long as overall cache activity permits it. If the cache usage is very heavy, then there
may be very little or no benefit to using CFW, since the majority of write operations will be immediately
directed to disk.
• When reading data back from the work data sets, the read operations can complete at cache speed
rather than at disk speed, as long as the data to be read still resides in cache. If the required data does
not reside in cache, a disk access is required and no performance benefit results. The higher the cache
activity, the less the performance improvement with CFW, because it is more likely that written data will
be destaged to disk before it can be read back.
Operation
DFSORT's use of cache fast write for the work data sets can be controlled with installation option CFW,
and run-time option CFW or NOCFW on the DEBUG statement. Note that cache fast write can only be used
if the DFSORT SVC has been installed. Also note that some control units override the CFW setting and
decide when and when not to cache the data.
ICEGENER
DFSORT's ICEGENER facility allows you to more efficiently process IEBGENER jobs. Qualifying IEBGENER
jobs are processed by the equivalent, but more efficient, DFSORT copy function. If the DFSORT copy
function cannot be used (for example, IEBGENER control statements are specified), control is
automatically transferred to the IEBGENER program.
If your site has installed ICEGENER as an automatic replacement for IEBGENER, you need not make any
changes to your jobs to use ICEGENER. If your site has not chosen automatic use of ICEGENER, you can
use ICEGENER by substituting the name ICEGENER for IEBGENER on any EXEC statement or LINK macro
call.
The use of ICEGENER rather than IEBGENER for copy applications can result in significant savings in
elapsed time, CPU time, and EXCP counts. Figure 1 on page 10 shows an example of the tremendous
performance advantages of ICEGENER over IEBGENER.
IDCAMS BLDINDEX
DFSORT provides support that enables IDCAMS BLDINDEX to automatically use DFSORT to improve the
performance of most BLDINDEX jobs that require BLDINDEX external sorting.
While DFSORT automatically optimizes many of its tuning decisions to improve performance, the
environment in which it runs affects its overall performance and, therefore, the programs and applications
that use DFSORT. DFSORT is designed to take advantage of the major components of the z/OS
environment.
The purpose of this chapter is to describe how DFSORT modifies its operation to take advantage of these
components and the benefits it derives from them.
The majority of information in this chapter applies only to sorting applications. While copying and merging
are also commonly used features of DFSORT, it is sorting that uses the most resources and is the most
important to tune.
This chapter includes the following information:
• Storage hierarchy
• Virtual storage
• System-managed storage
Storage hierarchy
Within a z/OS environment, data resides across a storage hierarchy. Each level of this hierarchy is
represented in hardware by a different component, and the amount of each component in a system is
usually variable between some minimum and maximum levels. A typical representation of these
components (from top to bottom) would be:
1. Processor cache
2. Central storage
3. Storage control cache
4. Disk
5. Tape
The components at the top of the hierarchy (processor cache, central storage) are somewhat expensive
on a per-byte basis (and thus relatively small in capacity), but have very fast access times. In contrast, the
components at the bottom of the hierarchy (disk, tape) are relatively inexpensive and have very large
capacities, but their access rates are considerably slower. As you go from top to bottom in the hierarchy,
the components typically get less expensive per byte, have higher capacities, and have slower access
times.
DFSORT attempts to take advantage of all the levels of the storage hierarchy. The following sections
briefly describe how DFSORT accomplishes this.
Processor cache
Processor cache is the special high-speed memory from which processors access their instructions and
data. This memory has a much faster access rate than central storage, and is an integral part of IBM
processors. It contains a copy of those portions of central storage that have been recently referenced.
When an instruction or piece of data is needed by the processor and is not in the processor cache, a
"cache miss" takes place and the processor waits while a copy of the data is brought into the cache. This
results in higher CPU times.
DFSORT is designed to make efficient use of the processor cache by reducing cache misses as much as
possible.
Central storage
Of all the components that affect DFSORT's performance, central (or main), storage is the most crucial.
Sorting is a memory-intensive operation. Without enough central storage to back the virtual storage needs
of DFSORT, its performance (as well as the performance of other applications on the system) degrades
significantly due to excessive system paging activity.
To sort efficiently, DFSORT needs large amounts of virtual storage. Its needs grow with the amount of data
being sorted; a data set four times as large as another requires roughly twice as much virtual storage to
sort with the same degree of efficiency. If this virtual storage is not backed by central storage when
DFSORT is running, there is a noticeable performance degradation on the system.
Central storage also plays an important role in the use of Hipersorting, dataspace sorting, and memory
object sorting. DFSORT bases its use of these very efficient sorting methods on the amount of central
storage it can use without causing excessive paging on the system. If too much central storage paging
would result, DFSORT limits its use of these methods and relies on work data set I/O to auxiliary storage.
See “Hipersorting” on page 37, “Memory object sorting” on page 35, and “Sorting with data space” on
page 40 for more information about exploiting central storage to reduce I/O and improve elapsed time.
Disk
In most cases, DFSORT's Blockset technique adjusts automatically to take advantage of the geometry of
the particular IBM disk being used. This is especially true for work data sets, whose block sizes and
distribution of data play crucial roles in the performance of DFSORT.
The location of input, output, and work data sets, as well as the speed of the disks on which they reside
has a significant effect on the performance of a sort application. For best results, work data sets should be
placed on different storage subsystems than the input and output data sets. This helps avoid channel,
control unit path, and device contentions. To attain the maximum performance benefit, these data sets (or
at least the input and output data sets) should be placed on the fastest disks so that DFSORT can take
advantage of their speed.
In general, while DFSORT has no control over where its data sets are allocated, it does automatically
optimize its access patterns based on data set location to achieve the best possible performance.
When allocating DFSORT work data sets on devices attached to non-synchronous storage control units or
connected to ESCON channels, elapsed time may be degraded for certain applications. To avoid this
degradation, it is especially important to follow the virtual storage guidelines described in “Virtual storage
guidelines” on page 43 and to ensure that DFSORT has an accurate knowledge of the size of the data set
being sorted. See z/OS DFSORT Application Programming Guide for more details about non-synchronous
storage subsystem considerations.
In addition to data set location, certain data set characteristics, such as block size, are also very
important when considering performance. As mentioned before, DFSORT automatically chooses an
optimal block size for its work data sets. Using DFSORT's installation option SDB=LARGE, SDB=INPUT or
SDB=YES enables DFSORT to choose optimal block sizes for its output data sets on disk as well. Of less
importance, using installation option SDBMSG=YES enables DFSORT to choose optimal block sizes for its
message data sets on disk.
Tape
Tape is the least expensive media on a per-byte basis. It has the highest capacity for data storage of any
component in the storage hierarchy. But it also has a relatively slow access time and must be accessed
sequentially. However, automatic tape libraries and IDRC compacted tape make tape a good choice for
SORTIN and SORTOUT. Further, if these tape data sets are managed by DFSMSrmm or a tape
management system that uses ICETPEX, DFSORT can obtain important information such as input file size,
and input and output attributes, that help DFSORT optimize performance.
Tape devices are not recommended for use as work data sets. Since a fast access rate is critical for work
data sets, and work data tends to be accessed in a nonsequential manner, performance is significantly
better when disk devices are used as work data sets rather than tape devices. In fact, tape work data sets
should be avoided if at all possible, because they prevent DFSORT from using its efficient Blockset
technique.
Certain data set characteristics, such as block size, are important when considering performance on tape
devices. Using DFSORT's installation option SDB=LARGE or SDB=INPUT enables DFSORT to choose
optimal block sizes greater than 32K for its output data sets on tape. Using installation option SDB=YES
enables DFSORT to choose optimal block sizes up to 32K for its output data sets on tape. Of less
importance, using installation option SDBMSG=YES enables DFSORT to choose optimal block sizes up to
32K for its message data sets on tape as well
Virtual storage
Logically, DFSORT (like any other application) works within a virtual address space. Installation defaults
such as TMAXLIM and run-time options such as REGION and MAINSIZE determine the size of this
address space. With the exception of dataspace sorting (see “Memory object sorting, hipersorting and
data space sorting” on page 6 for a discussion of dataspace sorting) and memory object sorting (see
“Memory object sorting, hipersorting and data space sorting” on page 6 for more information about
memory object sorting), this size remains constant throughout most of the sorting process.
DFSORT attempts to make the best use of the virtual storage it has available. If you provide DFSORT with
enough virtual storage, it might be able to sort the input data set entirely in virtual storage (an "in-main-
storage" sort) without need of work data sets or Hiperspace. This is the preferred method of sorting small
data sets up to a few megabytes in size. To sort larger input data sets, DFSORT may use data space or
memory objects to perform an "in-main-storage" sort.
When an in-main-storage sort is not possible or practical, DFSORT processes a portion of the input data
set at a time, then combines all of these processed portions together into the final sorted data set. It is
here that DFSORT excels at allocating virtual storage effectively in order to minimize both the number of
portions as well as the time spent combining the portions into the output data set.
With dataspace sorting, DFSORT creates a data space to help carry out its processing. This is a new area
of virtual storage, and is in addition to the original (specified or defaulted) virtual storage requested. The
size of the data space is sufficient to guarantee an efficient sort (or dataspace sorting is not used).
In addition, DFSORT adjusts the size of the data space during processing, as necessary, in response to
system paging levels. When system paging levels rise, DFSORT reduces its use of virtual storage (as long
as this reduction does not significantly degrade DFSORT performance).
Environment considerations 15
Environment Considerations
DFSORT also tries to move as many of its data areas above 16 MB virtual as it can to help provide virtual
storage constraint relief for the system.
With z/OS, the MVS™ address space expands up to 16 exabytes in size. The architecture that creates this
address space provides 64-bit addresses. The 64-bit address space marks the 2-gigabyte virtual line
called "the bar". The bar separates storage below the 2-gigabyte address, called "below the bar", from
storage above the 2-gigabyte address, called "above the bar". DFSORT obtains storage in virtual storage
above the bar as a "memory object" for sorting.
Main storage
Main storage is the portion of virtual storage in the primary address space that DFSORT limits itself to
using. In general, the more main storage you make available to DFSORT, the better the performance for
larger jobs. To prevent excessive paging, insure that sufficient real storage is available to back up the
amount of main storage used. This is especially important with main storage sizes greater than 32 MB.
The default amount of main storage that will be made available to DFSORT is defined when it is installed.
Although it is possible to run a "very minimal" sort application in 88 KB of storage, the actual minimum
amount of storage required is generally 128 KB to 440 KB depending on the application. The
recommended minimum to avoid severe performance degradation is about 6MB but a greater amount
may be required for optimal performance when sorting large files. Guidelines for setting these values are
given in Table 4 on page 43 and Table 5 on page 44. It is recommended that the user not override the
mainsize storage amount, but rather allow DFSORT to adjust the storage up to the DSA limit (DFSORT only
makes this adjustment for a sort application when doing so can improve performance).
System-managed storage
The Storage Management Subsystem (SMS) makes data set allocation very easy and efficient. Having SMS
manage the temporary data sets can be a good first step in migrating to system-managed storage.
However, the SMS automatic class selection (ACS) routines can unintentionally affect DFSORT data set
allocations and job performance, so you might need to coordinate changes to the ACS routines with the
site's storage administrator, as described here.
When any data set is allocated on an SMS system, the allocation request passes through the system's
data class and storage class ACS routines. If the data set will be system-managed, the request also
passes through the system's management class and storage group ACS routines. There is only one set of
ACS routines per site, and they are very powerful. They can override DFSORT installation options, such as
VIO=NO, and can even override requests for a certain data, storage, or management class, or a certain
unit or volume.
It is important to use &maxsize rather than &size in the ACS routines for volume assignment decisions
related to the size of DFSORT work data sets. &size only takes into account the primary allocation,
whereas &maxsize takes into account both the primary and possible total secondary allocation. Thus, the
use of &maxsize will allow for a more accurate decision on the assignment of unit, storage class, storage
group, and so on.
As an example of how ACS routines can affect the performance of DFSORT applications, consider a
storage class ACS routine that assigns all temporary data sets to the STANDARD storage class, as shown
in Figure 2 on page 17. The storage class is then used to assign the temporary data sets to a storage
group. Because the routine does not differentiate between DFSORT temporary data sets and other
temporary data sets, the storage group ACS routine cannot selectively prevent DFSORT temporary data
sets from being assigned to VIO. Allocating temporary data sets to VIO works well in most cases, but
might not be desirable for DFSORT temporary data sets, as explained in “VIO for DFSORT data sets ” on
page 48.
PROC 1 STORCLAS
......
......
SELECT
WHEN(&DSTYPE = 'TEMP')
SET &STORCLAS = 'STANDARD'
END
......
......
END /* END OF PROC */
One way to avoid allocating DFSORT temporary data sets to VIO is to write the ACS storage class routine
so it assigns all DFSORT temporary output and work data sets (for example, those with ddnames
SORTOUT, SORTOFdd, SORTWKdd, and SORTDKdd) to a special NONVIO storage class. Using the &DD
variable is the most efficient way to determine whether or not a data set is a DFSORT data set. Because
you cannot use the &DD variable to check the ddname in the storage group ACS routine, you must check
for DFSORT temporary data sets in the storage class ACS routine as shown in Figure 3 on page 17.
PROC 1 STORCLAS
SELECT
WHEN(&DSTYPE = 'TEMP')
IF (&DD = &DFSORTDD)
THEN SET &STORCLAS = 'NONVIO'
ELSE SET &STORCLAS = 'STANDARD'
OTHERWISE SET &STORCLAS = '
END
......
......
END /* END OF PROC */
The storage group ACS routine can then look for the SORT storage class, and assign the data sets to a
non-VIO storage group, as shown in Figure 4 on page 18. The site's storage administrator will need to
create the special NONVIO storage class and alter the storage class and storage group ACS routines.
Environment considerations 17
Environment Considerations
PROC 1 STORGRP
......
......
/* ASSIGN ALL TEMPORARY DATA SETS THAT ARE NOT DFSORT */
/* DATA SETS TO 'SGVIO' */
SELECT
WHEN(&DSTYPE = 'TEMP' && &STORCLAS; ¬= 'SORT')
SET &STORGRP = 'SGVIO','PRIME'
Be aware that the ddnames SORTOUT, SORTWKdd, and SORTDKdd can be changed to other names when
a program calls DFSORT. If other DFSORT output and work data set ddnames are in common use at your
site, you should also include them in the ACS routines. This scheme does not work for OUTFIL data sets
specified with the FNAMES operand unless common ddnames are specified.
Note: Filtering on SORTIN in the ACS routines cannot prevent DFSORT temporary input data sets from
being allocated to VIO. These data sets are allocated in preceding steps and passed to DFSORT. Thus,
SORTIN is not the ddname used when these data sets are allocated.
If the only DFSORT temporary data sets are dynamically allocated work data sets, another way to avoid
VIO allocation is to use a non-VIO generic device type for the installation option DYNALOC and the run-
time option DYNALLOC. The storage group ACS routine could then be written to assign a pool storage
group to data sets with that generic device type. Again, the site's storage administrator will need to
establish a non-VIO generic device type and alter the storage group ACS routine.
Note: DFSORT's dynamic allocation uses a primary allocation of 0 and non-zero secondary extents. In this
way, space is only used when the data set is extended. If the attempts to extend the data sets fail or the
required space can still not be obtained, DFSORT may do one of two things:
• delete and then reallocate a dynamically allocated work data set using a non-zero primary allocation to
force selection of a volume with enough free space to satisfy the request or
• retry the allocation using a device type matching the unit type of the original allocation (for example,
3390) in an effort to allocate the required space in a volume from another pool storage group. This
behavior, though rare, should be taken into account when designing ACS routines that make
assignments based solely on device type.
There might be other cases as well where the site's ACS routines unintentionally alter DFSORT
performance or data set allocation. Be aware of this, and if you encounter problems, consult with your
storage administrator to work out a joint solution.
Software that changes disk space allocations or adds additional volumes should be excluded from
changing DFSORT work data set space. DFSORT's dynamic allocation function calculates the optimal work
space for each sort application, adjusts the allocated work space as needed, and terminates if the
required space is not available. Software that reduces DFSORT's disk work data set allocations may cause
a sort application to fail. To avoid this situation, you should take the appropriate steps to have such
software exclude the following ddnames from being changed: SORTWK*, STATWK*, DATAWK*, DAnnWK*,
STnnWK*, SWnnWK* and ccccWK* (where cccc is defined with the SORTDD=cccc option). Additionally if
you are using DFSMS data classes, you should not assign those ddnames to a DFSMS data class that uses
the Space Constraint Relief feature.
IEBGENER” on page 20, DFSORT's use greatly increases, making it even more important to run DFSORT
resident.
5
ASCII tapes had the following parameters:
(LABEL=AL or OPTCD=Q) and RECFM=D and BUFOFF¬=L
or
(LABEL=AL or OPTCD=Q) and RECFM¬=D and BUFOFF¬=0
6
An attempt to read the DSCB for the SYSUT1 data set caused an error.
7
An attempt to read the DSCB for the SYSUT2 data set caused an error.
8
The SYSUT1 data set had keyed records.
9
User labels were present.
Under such circumstances, DFSORT transfers control to IEBGENER. However, IEBGENER may not be able
to process the copy application either.
See z/OS DFSORT Installation and Customization for complete details on installing ICEGENER as an
automatic replacement for IEBGENER.
Storage options
By using appropriate values for DFSORT installation storage options, you can ensure that the majority of
applications have sufficient virtual storage. If the IBM supplied installation default value for an installation
storage option is inappropriate for the majority of DFSORT applications at your site, you should change it
to a more appropriate value. For specific applications, the installation values can be overridden using run-
time options. See z/OS DFSORT Installation and Customization and z/OS DFSORT Application Programming
Guide for details about DFSORT storage options and the relationships between the options.
Recommendations
You must provide sufficient virtual storage to DFSORT using the guidelines given in “Virtual storage
guidelines” on page 43 and “Virtual storage and sorting with data space or memory objects ” on page
44.
The following installation storage option values are recommended:
SIZE
The default, SIZE=MAX, is recommended. This enables DFSORT to use as much virtual storage as
possible, both above and below 16 MB virtual, subject to the limits set by MAXLIM and TMAXLIM.
When installation option SIZE=MAX, EXEC PARM option SIZE=MAX, or run-time option
MAINSIZE=MAX is in effect, the TMAXLIM, RESALL, and RESINV options are used. These options are
not used when SIZE=n or MAINSIZE=n is in effect.
You should also be aware of how the JCL REGION parameter can affect DFSORT virtual storage
allocation. While subject to the constraints of your site's IEFUSI and IEALIMIT exits, the JCL REGION
value limits the amount of below 16 MB virtual storage.
DFSORT attempts to place as much of its storage as possible above 16 MB virtual. DFSORT, however,
still needs sufficient storage below 16 MB virtual to run effectively. In general, a REGION value of at
least 512 KB is best. If DFSORT is called by a program, the REGION value should be large enough to
allow sufficient storage for DFSORT and the program. If E15 or E35 user exits are used, a larger
REGION value might improve performance because these functions use more storage below 16 MB
virtual.
In general, the minimum of:
Installation considerations 21
Installation Considerations
One way to improve the performance of these applications is to raise the MINLIM value (for example,
to the MAXLIM value). This enables such applications to run with a larger amount of virtual storage.
This strategy, however, might cause some of these applications to fail due to insufficient reserved
storage. Using run-time options, you should change the failing applications to use SIZE=MAX or
MAINSIZE=MAX and set appropriate values for user exit sizes (on the MODS control statement),
RESALL, and RESINV.
OVERRGN
The default and recommended values for this option are 64 KB for directly-invoked and 16 KB for
program-invoked applications.
RESALL
The default is 4 KB and should normally not be modified. If a storage related failure occurs when
sufficient virtual storage (REGION and SIZE) has been specified, try setting RESALL to a larger value to
correct the problem.
RESINV
Normally, a RESINV value of 16 KB is sufficient and is recommended.
ARESALL
ARESALL is seldom needed and can be kept at its default of 0.
ARESINV
ARESINV is seldom needed and can be kept at its default value of 0.
See z/OS DFSORT Installation and Customization for more information about these parameters.
Recommendations
The recommended installation settings for Hipersorting, memory object sorting, and data space sorting
are the IBM-supplied defaults:
• EXPMAX=MAX
• EXPOLD=50%
• EXPRES=10%
• TUNE=STOR
• HIPRMAX=OPTIMAL
• MOSIZE=MAX
• MOWRK=YES
• DSPSIZE=MAX
These settings are described in detail in the sections that follow.
EXPMAX=MAX
This setting allows maximum Hipersorting, memory object sorting, and data space sorting activity on
a system, especially during periods when applications (both DFSORT and non-DFSORT) are making
little use of central storage. Setting EXPMAX=n, where n is a number of megabytes, or EXPMAX=p%,
where p% is a percentage of the configured central storage, should only be considered when it is
important for a site to limit the total amount of storage used by all DFSORT applications on the
system.
Installation considerations 23
Installation Considerations
Setting EXPMAX to n or p% can help prevent situations where a long running sort application holds
storage resources that may be needed by new work entering the system later. Note that this only
limits DFSORT's storage usage. The total storage used by DFSORT and other address spaces can
exceed EXPMAX depending on the values in effect for EXPRES and EXPOLD.
EXPOLD=50%
This setting is recommended only if migration of storage data to auxiliary storage does not cause a
problem on your system. During batch processing windows, migrating old data to auxiliary storage can
improve overall system performance, since it allows more active data, like DFSORT work data, to use
storage in place of disk work data sets. But, the one-time migration can have a negative impact on
system performance, especially if the paging subsystem is not large enough to hold the number of
frames that are migrated. Using EXPOLD=50%, along with TUNE=STOR, causes DFSORT to continually
examine the amount of old storage and never allow a sort to cause more than half of it to be used by
DFSORT.
If the migration of large amounts of old storage data is not a concern and you want to maximize
DFSORT's use of central storage, consider setting EXPOLD=MAX. This setting should only be used if
there is a robust paging subsystem large enough to support a large spike in the number of frames
being migrated to auxiliary storage.
If the migration of large amounts of old storage data is a concern for your site, set EXPOLD to a small
number or zero. Setting EXPOLD=0 limits DFSORT to only consider available frames in its evaluation of
resources for Hipersorting, memory object sorting or data space sorting.
EXPRES=10%
This setting permits DFSORT almost full use of available storage when non-DFSORT activity is light,
but greatly reduces the possibility of an overuse of storage when there is a sudden increase in the use
of storage by non-DFSORT applications. If you need to keep central storage available for non-DFSORT
applications, increase this value. If you want DFSORT to have full use of available storage, set
EXPRES=0.
TUNE=STOR
This setting causes DFSORT to allocate central storage in increments sized to balance usage when
multiple sorts are executing concurrently on the same system. DFSORT continually examines
available resources and dynamically makes adjustments to the increment size. If EXPMAX, EXPOLD or
EXPRES are specified as percentages, DFSORT will also recalculate those values based on available
resources. DFSORT's dynamic allocation will increase the space allocations to reduce the risk of
failure if central storage resources become constrained during execution of a sort.
If conserving disk work space is more important in your environment, you can use TUNE=DISK which
will cause each sort to allocate all of the available work space it plans to use immediately. This can
cause a small number of sorts to monopolize available central storage but DFSORT's dynamic
allocation will be able to more aggressively reduce the disk work space allocations based on the
expected central storage usage.
Note that even sorts which use JCL or pre-allocated work data sets will allocate storage based on the
TUNE parameter. This is necessary so that all sorts compete equally for the same storage. Similarly,
this is why TUNE cannot be overridden as a run-time option. TUNE=DDYN can be used to cause sorts
that use DFSORT's dynamic allocation to allocate all of the available storage they plan to use
immediately, and cause sorts using JCL or pre-allocated work data sets to allocate in increments. This
is useful for special situations where dynamic allocation is frequently used by the majority of sorts,
and optimizing disk space is a priority, but there might be a known time period when the majority of
sorts do not use dynamic allocation and would benefit from balancing central storage.
TUNE=OLD is available for customers who prefer that DFSORT allocate central storage in fixed
increment sizes. If EXPMAX, EXPOLD or EXPRES is specified as a percentage, DFSORT calculates that
value once based on configured central storage.
HIPRMAX=OPTIMAL, MOSIZE=MAX, DSPSIZE=MAX, MOWRK=YES
These settings allow the DFSORT installation options EXPMAX, EXPOLD, and EXPRES to control the
total Hipersorting, memory object sorting, and data space sorting activity on a system.
HIPRMAX=n or HIPRMAX=p% should be reserved for use as a run-time override for applications that
have a special reason to limit the amount of Hiperspace available for Hipersorting.
MOSIZE=n or MOSIZE=p% should be reserved for use as run-time overrides for applications that have
a special reason to limit the amount of virtual storage available for memory object sorting.
MOWRK=NO should only be used for installations that have a special need to only allow memory
objects to be used as an extension of main storage instead of intermediate work space.
DSPSIZE=n or DSPSIZE=p% should be reserved for use as run-time overrides for applications that
have a special reason to limit the amount of virtual storage available for data space sorting.
When Hiperspace, memory object and data space usage is limited, JCL work data sets (SORTWKdd)
may no longer be large enough to complete a sort, or the installation DYNSPC default may not be large
enough for unknown file size conditions. As a result, error messages (for example, ICE046A and
ICE083A) may be issued due to less central storage being used. The use of dynamic allocation with an
appropriately large installation DYNSPC value is recommended. If you have JCL work data sets and
you want DFSORT to ignore them, use DYNAUTO=IGNWKDD.
start iceopt,iceprm=01
Installation considerations 25
Installation Considerations
ICEPRMxx members in PARMLIB are the recommended way to change your installation defaults. You can
activate different ICEPRMxx members for different LPARs.
Alternatively, you can use the ICEMAC macro and a USERMOD to update the source distribution library to
reflect the changed options.
See z/OS DFSORT Installation and Customization for complete details on installation options and defaults.
Note: The supplied DFSORT installation defaults were chosen to balance DFSORT versus system
performance and resource usage, as well as to have DFSORT operate in a manner appropriate for most
customers. Therefore, the supplied defaults should only be changed on an exception basis.
Installation defaults that are inappropriate for an application should be overridden for that application
with the corresponding run-time options. See z/OS DFSORT Application Programming Guide for complete
details of run-time options.
Installation considerations 27
Installation Considerations
Installation considerations 29
Installation Considerations
Installation considerations 31
Installation Considerations
See z/OS DFSORT Installation and Customization for a complete list of site-wide installation options. See
z/OS DFSORT Application Programming Guide for corresponding run-time overrides.
Installation exits
DFSORT allows you to use user-written, installation-wide initialization and termination exit routines to
perform a variety of functions, such as overriding the options currently in effect and collecting statistical
data. For tuning purposes it is often advantageous to install these exits for the following reasons:
• These exits can be used for performance data gathering to help you understand the use of DFSORT at
your site and make the appropriate tuning decisions based on this information.
• An initialization routine allows you to override the run-time values set for certain options, which
enforces your decisions for those option values for all DFSORT applications at your site.
ICEIEXIT
A site-supplied ICEIEXIT routine can exercise control over certain DFSORT run-time options.
If present and activated, the ICEIEXIT routine is called and passed installation and run-time information
by DFSORT. The ICEIEXIT routine can then use current DFSORT and system information to determine
whether to change certain options in effect. This also permits site-wide control of certain options whose
installation defaults have been overridden at the application-level.
An ICEIEXIT routine can examine installation and run-time information related to:
• Storage limits
• Hiperspace limits
• Data space limits
• Memory object limits
• Use of VERIFY
• OUTFIL buffer space limits
and additional run-time information related to:
• DFSORT technique used
• Type of DFSORT application
• Method of DFSORT invocation
• Storage above 16 MB virtual
• Configured expanded storage
An ICEIEXIT routine can also change certain run-time options including:
• Storage limits
• Hiperspace limits
• Data space limits
• Memory object limits
• Use of VERIFY
• OUTFIL buffer space limits
A site could use an ICEIEXIT routine to control applications and enforce site standards. For example:
• Many options (for example, MAXLIM, SIZE, TMAXLIM, DSA) can affect the virtual storage used by
DFSORT. An ICEIEXIT routine could specify the amount of virtual storage to be used depending on such
factors as performance requirements and jobname.
Note: The time-of-day installation modules allow you to specify the virtual storage to be used
depending on the day and time when an application runs. See “Time-of-Day installation environments”
on page 26 for more information.
• Before creating a data space, DFSORT checks to see how much central storage either is not being used
or has gone unreferenced for a sufficient period of time. This is to make sure enough real storage is
available to back the data space without causing excessive system paging activity. An ICEIEXIT routine
can further reduce the risk of overcommitting central storage by limiting the amount of data space that a
single DFSORT application can use. This would also override any run-time specifications that try to get
around the installation default.
See z/OS DFSORT Installation and Customization for information about coding an ICEIEXIT routine and a
sample ICEIEXIT routine, which shows how the storage available to DFSORT can be dynamically modified
based on the jobname/stepname and type of application.
ICETEXIT
If present and activated, the ICETEXIT routine is called at the end of DFSORT application processing. It is
available for those who wish to make a thorough analysis of DFSORT performance data using a single
source of information. See “Using ICETEXIT data ” on page 70 for more information about using this
routine; see z/OS DFSORT Installation and Customization for complete information on how to write and
install an ICETEXIT routine.
Installation considerations 33
Installation Considerations
This chapter offers advice about improving the performance of individual DFSORT applications by using
run-time options related to the following areas:
• Memory object sorting
• Hipersorting
• Sorting with data space
• Cache fast write
• File size
• Storage
• Input and output data sets
The last section includes a table with information on run-time options available with DFSORT that can
affect performance.
Limitations
The maximum amount of memory object storage used by DFSORT is limited to the minimum of the
following values:
• The MEMLIMIT parameter limit on the total size of memory objects that can be allocated in a single job
step. See z/OS MVS JCL Reference for a description of the MEMLIMIT parameter.
• The IEFUSI exit limit on the total size of memory objects that can be allocated in a single job step. See
z/OS DFSMS Installation Exits for a description of IEFUSI.
• The MOSIZE value in effect, when set to a value other than MAX. The MOSIZE value in effect is either
the installation default, an overriding value specified at run-time, or an overriding value specified in the
installation ICEIEXIT routine. Note that the value specified in the ICEIEXIT routine overrides any other
value.
• Available central storage. Throughout the run, DFSORT checks the amount of available central storage.
If, as a result of any such check, either a central storage shortage is predicted, or one of the site limits
for total central storage usage by all memory object sorting applications is reached, DFSORT switches
from using a memory object to using disk work data sets.
In addition, all future DFSORT applications are prevented from using memory object sorting until the
central storage situation is relieved. This prevents memory object sorting applications by themselves
from causing a shortage of central storage.
The following are those cases for which you should not attempt to adjust your application; in these cases
the best performance for the individual job and for the system is achieved by not using memory object
sorting:
• Other performance features are in use: Memory object sorting is not used when DFSORT decides to
use Hipersorting or data space sorting. DFSORT dynamically chooses between using memory object
sorting, data space sorting, and Hipersorting and selects the one that provides the best performance for
the particular sort. Messages ICE180I and ICE188I indicate whether Hipersorting or data space sorting
was used for a particular run.
• The size of the input data set is very small: If the amount of data to be sorted is known to be small
enough that the sort can be accomplished in main memory, memory object sorting is not used.
Application adjustments
The following are those cases for which you may want to adjust your application in order to take
advantage of memory object sorting.
• The Blockset technique was not selected: Memory object sorting is supported only for the Blockset
technique. If Blockset is not selected, message ICE800I indicates why it was not selected. Note that the
ICE800I message is printed only when a SORTDIAG DD statement was coded in the sort’s JCL, or
installation option DIAGSIM=YES has been specified for your site. Use the ICE800I reason code to
determine the exact condition that is preventing the use of Blockset. If you are interested in using
memory object sorting for the job, change your application appropriately to eliminate the particular
condition, so that Blockset can be used.
• Insufficient available virtual storage: In some cases, the amount of virtual storage available to
DFSORT can influence the potential effectiveness of memory object sorting. The third value in message
ICE092I or ICE093I indicates the amount of storage available for a particular sort job. To help reduce
the likelihood of not using memory object sorting because of insufficient virtual storage, ensure that this
value is at least the minimum recommendation given in Table 5 on page 44.
• Insufficient available above the bar virtual storage: The MEMLIMIT parameter on the JOB or EXEC
JCL statement limits the total amount of memory object storage that can be allocated in a single job
step. For more effective use of memory object sorting, specify a MEMLIMIT value that is at least equal
to the size of the sorted data set, or specify an unlimited size for memory objects using the
MEMLIMIT=NOLIMIT parameter on the JOB or EXEC JCL statement. Note that if MEMLIMIT is not
specified, but REGION=0K or REGION=0M is specified on the JOB or EXEC JCL statement, then
MEMLIMIT=NOLIMIT is implied.
• Insufficient available central storage: The total amount of memory object storage DFSORT needs is
directly related to the size of the input data set. Memory objects are backed by central storage.
Therefore, insufficient available central storage for backing DFSORT’s memory objects has an effect on
the performance of memory object sorting. If the total amount of memory object storage DFSORT needs
would exceed the available central storage, DFSORT chooses not to use memory object sorting.
If you would like memory object sorting to be used, there are several possible approaches you can take:
– Make sure that the MOSIZE value, the MEMLIMIT value, or the installation IEFUSI exit are not limiting
the application to a small amount of memory object storage. Setting MOSIZE=MAX (or setting
MOSIZE to a very large value), and having MEMLIMIT and IEFUSI allow the use of a memory object at
least the size of the input data set, will remove this limitation.
– Rerun the application when system activity, especially other concurrent memory object sorting and
Hipersorting activity, is lower so that more central storage is available for the sort.
– Reduce the size of the input data set, so that less central storage is required for the sort. For some
applications it is not necessary to sort all of the data, since only a subset is needed for processing.
For example, INCLUDE, OMIT, SKIPREC, or STOPAFT can significantly reduce the amount of
intermediate storage required by DFSORT. See z/OS DFSORT Application Programming Guide for more
details about these features of DFSORT.
– Ensure that DFSORT has accurate information about the input file size. DFSORT can automatically
estimate the file size for disk input data sets and tape data sets managed by DFSMSrmm or a tape
management system that uses ICETPEX. However, there are certain situations, which DFSORT
reports with message ICE118I, in which DFSORT cannot determine the file size. See "File Size and
Dynamic Allocation" in z/OS DFSORT Application Programming Guide for more information on these
situations, and what to do about them.
• Insufficient work data set usage: When it is possible to use a combination of memory object and disk
storage as intermediate work space, DFSORT must decide how best to use disk work data sets. Specify
generous extent sizes for disk work data sets, especially for secondary extents.
In general, DFSORT takes into account the effects of using memory objects on both the application’s
performance and the system’s performance when determining whether or not to use memory object
sorting. If either effect is not desirable, DFSORT chooses not to use memory object sorting.
See “Memory object sorting, hipersorting and data space sorting” on page 6 for information on the
benefits and operation of memory object sorting and “Hipersorting, memory object sorting, and data
space sorting” on page 23 for additional information on using memory object sorting effectively.
Hipersorting
Hipersorting is a DFSORT capability that uses Hiperspaces for sorting. For a more detailed description of
this capability, see “Memory object sorting, hipersorting and data space sorting” on page 6. The sections
that follow include information on how to use Hipersorting in the most efficient way at your site.
DFSORT selects the most appropriate mode (Hiperspace-only, Hiperspace-mixed, or disk-only mode) for
each particular run. Not every sort application can use Hipersorting, and even for those sorts that can use
Hipersorting, it may be more advantageous not to use it under certain circumstances. This section
examines the most common reasons for not using Hiperspace and explains the possible actions that can
be undertaken to allow more jobs to take advantage of Hipersorting.
DFSORT adjusts its use of hiperspaces appropriately taking into consideration that real storage is a
resource available to all users of the system.
Some customers have expressed concerns that they would like to see Hipersorting used more often.
However, the use of hiperspace storage must always be weighed against the possibility of degrading
performance for a job or for the entire system, by overusing storage. If DFSORT were to use Hipersorting
indiscriminately, there could be a significant increase in paging activity and a resulting reduction in total
system performance, affecting all jobs, including sorts.
The recommended setting for HIPRMAX is OPTIMAL. With dynamic Hipersorting, DFSORT has full control
over total Hipersorting activity, and sites can customize their definition of HIPRMAX=OPTIMAL with the
installation options EXPMAX, EXPOLD, and EXPRES. HIPRMAX=n and HIPRMAX=p% should be reserved
for use as a run-time override for applications that have a special reason to limit the amount of
Hiperspace available for Hipersorting.
Run-time considerations 37
Run-Time Considerations
The number of kilobytes of Hiperspace storage used during the sort is displayed in message ICE180I. If
Hipersorting is not used, the message shows a value of 0K.
Limitations
The maximum amount of Hiperspace used by DFSORT is limited to the minimum of the following values:
• The IEFUSI exit limit on the total amount of Hiperspace and data space that can be allocated in a single
job step. See z/OS MVS Installation Exits for a description of IEFUSI.
• The HIPRMAX value in effect, when set to a value other than OPTIMAL. The HIPRMAX value in effect is
either the installation default, an overriding value specified at run-time, or an overriding value specified
in the installation ICEIEXIT routine. Note that the value specified in the ICEIEXIT routine overrides any
other value.
• Available storage. Throughout the run, DFSORT determines the pages available on the system, subtracts
from this the amount of storage needed by other concurrent Hipersorting applications, and factors in
the values specified for installation options EXPMAX, EXPOLD, and EXPRES. If as a result of any such
check, either a storage shortage is predicted or one of the site limits for total storage usage by all
Hipersorting applications is reached, DFSORT switches from using Hiperspace to using disk work data
sets for all currently running Hipersorting applications.
In addition, all future DFSORT applications are prevented from using Hipersorting until the storage
situation is relieved. This prevents Hipersorting applications by themselves from causing excessive
paging.
Since this last criteria depends very heavily on system activity, especially other concurrent Hipersorting
and memory object sorting activity, DFSORT applications can use varying amounts of Hiperspace when
run at different times and under different conditions. In fact, it is possible for such applications to not
use any Hiperspace.
The following are those cases for which you should not attempt to adjust your application; in these cases
the best performance for the individual job and for the system is achieved by not using Hipersorting:
• Other performance features are in use. Hipersorting is not used when DFSORT decides to use
dataspace or memory object sorting. DFSORT dynamically chooses between using dataspace sorting,
memory object sorting and Hipersorting; DFSORT selects the one that provides the best performance
for the particular sort. Messages ICE188I and ICE199I indicate whether dataspace sorting or memory
object sorting was used for a particular run.
• The size of the input data set is very small. If the amount of data to be sorted is known to be small
enough that the sort can be accomplished in main memory, Hipersorting is not used. Since no
intermediate data is generated, neither Hiperspace nor disk work data sets are needed. The presence of
message ICE080I indicates that a sort was processed in main memory.
Application adjustments
The following are those cases for which you may want to adjust your application in order to take
advantage of Hipersorting.
• The Blockset technique was not selected. Hipersorting is supported only for the Blockset technique. If
Blockset is not selected, message ICE800I indicates why it was not selected.
Note that the ICE800I message is printed only when a SORTDIAG DD statement was coded in the sort's
JCL, or installation option DIAGSIM=YES has been specified for your site. Use the ICE800I reason code
to determine the exact condition that is preventing the use of Blockset. If you are interested in using
Hipersorting for the job, change your application appropriately to eliminate the particular condition, so
that Blockset can be used.
• Insufficient available virtual storage. In some cases, the amount of virtual storage available to
DFSORT can influence the potential effectiveness of a Hiperspace. A Hiperspace of a certain size could
be too small to improve performance when an insufficient amount of virtual storage is available,
whereas the same size Hiperspace might be large enough to improve performance when a sufficient
amount of storage is available. Since DFSORT does not use Hiperspace when doing so would not result
in a performance benefit, insufficient virtual storage can indirectly prevent the use of Hipersorting.
Supply DFSORT with sufficient virtual storage if you would like Hipersorting to be used. The third value
in message ICE092I or ICE093I indicates the amount of storage available for a particular sort job. To
help reduce the likelihood of not using Hipersorting because of insufficient virtual storage, ensure that
this value is at least the maximum recommendation given in Table 4 on page 43. If necessary, increase
the amount of virtual storage available to the job by specifying a larger MAINSIZE value on the OPTION
control statement and/or raising the REGION value on the sort step EXEC statement.
• Insufficient available central storage. The size of the input data set in relation to the total available
central storage has an important effect on the performance of Hipersorting. If the size of the Hiperspace
that could be created is too small to hold a significant percentage of the intermediate data, then the
performance of the run would be degraded compared to using disk-only mode. Therefore, DFSORT
chooses not to use Hipersorting in this situation.
If you would like Hipersorting to be used, there are several possible approaches you can take:
– Make sure that the HIPRMAX value or the installation IEFUSI exit is not limiting the application to a
small amount of Hiperspace. Setting HIPRMAX=OPTIMAL (or to a very large value) and having IEFUSI
allow at least 2 GB of Hiperspace per application will remove this limitation.
– Make sure that the EXPMAX, EXPOLD, and EXPRES values allow significant amounts of Hipersorting.
This is accomplished by setting EXPMAX and EXPOLD to large values (or MAX) and EXPRES to a small
value.
– Rerun the application when system activity, especially other concurrent Hipersorting and memory
object sorting activity, is lower so that more storage is available for the sort. The more storage
available, the larger the Hiperspace that can be created by DFSORT, and the larger the data set size
for which Hipersorting can be allowed.
– In situations where business critical applications are executing concurrent with non-critical
applications, it may be desirable to limit Hiperspace usage by a non-critical sort application to leave
resources available for the business critical sort applications.
Remember that some data sets are so large that Hipersorting can never be used to sort them, even if
all of the storage installed on the system were available for the sort. To allow Hipersorting in such
cases, you can either break up the large sort into multiple smaller sorts, or install more storage on the
system.
– Reduce the size of the input data set, so that less storage is required for the sort. For some
applications it is not necessary to sort all of the data, since only a subset is needed for processing.
For example, INCLUDE, OMIT, SKIPREC, or STOPAFT can significantly reduce the amount of
intermediate storage required by DFSORT. See z/OS DFSORT Application Programming Guide for more
details about these features of DFSORT.
– Ensure that DFSORT has accurate information about the input file size. DFSORT can automatically
estimate the file size for disk input data sets and tape data sets managed by DFSMSrmm or a tape
management system that uses ICETPEX. However, there are certain situations, which DFSORT
reports with message ICE118I, in which DFSORT cannot determine the file size. See "File Size and
Dynamic Allocation" in z/OS DFSORT Application Programming Guide for more information on these
situations, and what to do about them.
– The parameters that control the system resources manager (SRM) can indirectly affect the amount of
storage that is available for all the jobs on your system, including sort jobs. For example,
PWSS=(0,100) may cause DFSORT to not use Hipersorting. See z/OS MVS Initialization and Tuning
Reference for information about SRM and its parameters.
• Inefficient work data set usage. When a Hiperspace-mixed mode run is possible, DFSORT must decide
how best to use both Hiperspace and disk work data sets. In most cases, trade-offs can be made such
that both types of intermediate storage can be used efficiently. In some cases, however, it is impossible
to use both Hiperspace and disk work data sets efficiently, in which case DFSORT chooses not to use
Hipersorting.
Run-time considerations 39
Run-Time Considerations
In order to avoid such cases, use only 3390 or later model disks, and supply sufficient virtual storage to
DFSORT, as described in Table 4 on page 43. Sometimes, it is necessary to rerun the jobs when there is
less system activity (to allow selection of Hiperspace-only mode) in order to take advantage of
Hipersorting.
In general, DFSORT takes into account the potential effects of using Hiperspace on both the application's
performance and the system's performance when determining whether or not to use Hipersorting. If
either effect is not desirable, DFSORT chooses not to use Hipersorting.
See “Memory object sorting, hipersorting and data space sorting” on page 6 for information on the
benefits and operation of Hipersorting and “Hipersorting, memory object sorting, and data space sorting”
on page 23 for additional information on using Hipersorting effectively.
See “Memory object sorting, hipersorting and data space sorting” on page 6 for information about the
benefits and operation of dataspace sorting, and “Hipersorting, memory object sorting, and data space
sorting” on page 23 for additional information on using data space sorting effectively.
File size
The input file size is important for sort applications, since it is used for several internal optimizations as
well as for dynamic work data set allocation. DFSORT can automatically estimate the file size for disk
input data sets and tape data sets managed by DFSMSrmm or a tape management system that uses
ICETPEX. However, there are certain situations, which DFSORT reports with message ICE118I, in which
DFSORT cannot determine the file size. See "File Size and Dynamic Allocation" in z/OS DFSORT
Application Programming Guide for more information on these situations, and what to do about them.
Storage
DFSORT sorts most efficiently when sufficient virtual storage is available to enable an optimal balance
between placing data in virtual and auxiliary storage. When virtual storage is limited, DFSORT must
expend more resources to transfer data between virtual and auxiliary storage, which causes increased
CPU time, elapsed time, and I/O usage.
Sufficient real storage must be available to support DFSORT's virtual storage requirements. Supplying
DFSORT with more virtual storage might not improve performance if the available system resources
cannot accommodate the corresponding increase in virtual storage activity. If real storage resources
become overcommitted, excessive paging can result. This can cause the performance of both DFSORT
and the system to degrade. It is important, therefore, to balance virtual storage resources supplied to
DFSORT with the overall system resource requirements.
DFSORT's Dynamic Storage Adjustment (DSA) feature can let DFSORT tune the right amount of virtual
storage for sort applications relieving, system and application programmers of the task. See “Dynamic
storage adjustment ” on page 9 for more information on the benefits and operation of DSA.
See “Virtual storage ” on page 15 for a description of virtual storage.
Run-time considerations 41
Run-Time Considerations
main storage sort. I/O or Hiperspace usage is increased, however, reflecting the need to write
intermediate data to work data sets or Hiperspace.
• When virtual storage is reduced further or the data set size is increased, DFSORT is forced to make less
efficient use of Hiperspace or work data sets. DFSORT does what it can to maintain performance but is
forced to use Hiperspace or work data sets less efficiently as the ratio of data set size to available
storage increases.The loss of efficiency adversely affects elapsed time and EXCP counts.
This performance degradation can be especially dramatic when using work data sets allocated on
devices attached to non-synchronous storage control units or connected to ESCON channels. In such
cases, it is especially important to follow the virtual storage guidelines explained in “Virtual storage
guidelines” on page 43.
• When virtual storage is very small or the data set size is very large, DFSORT may require several
additional passes over the data to perform the sort. This phenomenon is known as intermediate
merging. DFSORT issues message ICE247I to indicate intermediate merging was required; processing
continues with degraded performance. Figure 6 on page 42 shows the benefit of increasing virtual
storage to eliminate intermediate merging.
All other factors being equal, the range of data set sizes that DFSORT can sort efficiently (or sort without
requiring intermediate merging) grows roughly as the square of the virtual storage size. That is, doubling
the virtual storage in an application enables the application to handle data sets four times as large with
the same degree of efficiency. Likewise, halving the virtual storage causes the application to handle data
sets only one-fourth as large with the same efficiency.
Table 4: Recommended Minimum Storage Guidelines for Sorting Without Data Space.
Recommended Minimum
Input Data Set Size Storage
Less than 50 MB 4 MB
Run-time considerations 43
Run-Time Considerations
Table 4: Recommended Minimum Storage Guidelines for Sorting Without Data Space. (continued)
Recommended Minimum
Input Data Set Size Storage
50 MB to 100 MB 4-6 MB
100 MB to 200 MB 4-8 MB
200 MB to 500 MB 6-12 MB
500 MB to 1 GB 8-16 MB
1 GB to 2 GB 12-24 MB
More than 2 GB 16-32 MB
In order to guarantee the most efficient sorting, use the higher end of the range shown. In order to
guarantee efficient, but not necessarily optimum sorting, use the lower end. These values are intended for
sorting without data space. See Table 5 on page 44 for storage recommendations for sorting with data
space.
Although sort applications can usually run with less virtual storage than the recommended minimum, the
recommended amount enables DFSORT to sort most efficiently. Using less than the recommended
amount can result in the effects described in “Data set size and virtual storage ” on page 41.
Table 5: Recommended Minimum Storage Guidelines for Sorting with Data Space or Memory Objects.
Recommended Minimum
Storage for Dataspace Sorting
Input Data Set Size or Memory Object Sorting
Less than 200 MB 4 MB
200 MB to 500 MB 4-6 MB
500 MB to 1 GB 4-8 MB
1 GB to 2 GB 4-10 MB
More than 2 GB 4-12 MB
In order to guarantee the most efficient sorting, use the higher end of the range shown. In order to
guarantee efficient, but not necessarily optimum, sorting, use the lower end. These values are intended
for sorting with data space or memory objects. See Table 4 on page 43 for storage recommendations for
sorting without data space or memory objects.
Block sizes
Choosing an efficient block size can improve space utilization and I/O performance. DFSORT's
SDB=LARGE, SDB=INPUT or SDB=YES installation option allows DFSORT to automatically select the
system determined optimum block size for output data sets as long as you do not specify the BLKSIZE
explicitly. Use one of these installation options, as appropriate, for optimum space utilization and
performance. SDB=INPUT is the supplied default. See z/OS DFSORT Installation and Customization for
details about the SDB installation option values. You can use the SDB run-time option to override the SDB
installation option when appropriate for particular jobs.
Space utilization
The amount of space on a track or cylinder occupied by user data depends on the block size specified for
the data set. Grouping logical records into blocks reduces the amount of space needed to store data.
Because fewer physical records are needed to store the same number of logical records, the amount of
space for count and key areas, and for gaps between records is reduced.
Larger block sizes offer better opportunities for increased disk space utilization. An appropriately selected
block size can result in higher space utilization than a smaller block size. An inappropriately selected
block size (large or small) can result in poor space utilization.
Figure 7 on page 46 shows the 3390 space utilization with various block sizes for a fixed-blocked data
set with a logical record length of 160.
Run-time considerations 45
Run-Time Considerations
I/O performance
Although small block sizes permit more concurrent channel operations, they reduce the net data transfer
rate (the actual amount of data transferred per second). This can impact the elapsed time of a DFSORT
application performing a significant amount of I/O. Small block size transfer also requires more CPU
involvement and can, therefore, increase CPU time.
Large block sizes enable a higher net data transfer rate for sequential data sets, such as for input and
output, and reduce the amount of processor time needed to service a channel program. For tapes, larger
block sizes (up to 256 megabytes), provide the best performance and are recommended. Always use
system determined blocksizes (SDB) for the best utilization and performance. An example of the benefits
of appropriately large input and output data set block sizes is shown in Figure 8 on page 47.
Recommendations
When selecting a block size for input or output, consider these factors:
• Smaller data set sizes generally result in less efficient use of disk space.
• DFSORT applications that process data sets with small block sizes will generate higher EXCP counts and
probably increase CPU time.
Type of device
For optimal performance, use high performance devices such as IBM's Enterprise Storage Server (ESS) or
TotalStorage DS8000 for input, output, and work data sets to gain the advantages of higher data transfer
rates and multiple path access. Other ways of improving DFSORT processing of the input and output data
sets are as follows:
• Use multiple channel paths
• Allocate enough primary space for the output data sets to avoid the need for additional extents.
• Use separate devices for the input and work data sets, and for the output and work data sets. (DFSORT
application data sets that are accessed concurrently should reside on separate devices.)
Run-time considerations 47
Run-Time Considerations
Run-time considerations 49
Run-Time Considerations
See z/OS DFSORT Application Programming Guide for a complete list of run-time override options.
Most sites have many applications involving sorting. DFSORT is often used by these applications either
directly by invoking DFSORT with JCL, or indirectly by invoking DFSORT from a program. In particular, the
COBOL SORT statement, and the PL/I sort routines result in calls to DFSORT (see the appropriate
language documents for complete details).
The way in which COBOL interfaces with DFSORT depends on the use of COBOL features such as
FASTSRT, NOFASTSRT, USING, GIVING and INPUT and OUTPUT PROCEDUREs, and DFSORT features
such as COBOL exits and DFSORT control statements. In general, the features you use are dictated by the
needs of your application. But in many cases, an application can achieve its results using one or another
of these features, that is, you have a choice. This chapter describes some of the COBOL and DFSORT
features you can choose and the performance and productivity implications of doing so.
See the appropriate COBOL guide if you are not familiar with any of the COBOL features described in this
chapter. See z/OS DFSORT Application Programming Guide if you are not familiar with any of the DFSORT
features described in this chapter. See z/OS DFSORT: Getting Started for tutorials on DFSORT control
statements and features.
Note: Many of the techniques described in this chapter can also be used with other languages that can
call DFSORT such as PL/I and Assembler.
• DFSPARM and IGZSRTCD enable you to pass certain DFSORT run-time options (such as MSGDDN) that
are ignored in SORTCNTL.
• Using the COBOL special register SORT-CONTROL enables you to pass different IGZSRTCD data sets to
DFSORT when you have multiple SORT statements. The statements in IGZSRTCD are actually placed in
the parameter list in storage that COBOL passes when it calls DFSORT. A separate parameter list with
the appropriate control statements is passed for each call.
• SORTCNTL and DFSPARM enable you to pass DFSORT control statements without increasing the storage
used for the parameter list.
For our examples, we use SORTCNTL, although IGZSRTCD or DFSPARM would do just as well.
Performance
When FASTSRT is in effect for a COBOL sort, DFSORT reads the input data set and writes the output data
set rather than COBOL. This results in reductions in elapsed time, CPU time and EXCPs. An example of the
benefits of FASTSRT is shown in Figure 9 on page 53. For this comparison, the same COBOL sort with
USING and GIVING was run with FASTSRT and NOFASTSRT.
Application considerations 53
Application Considerations
*-----------------------------------------------------------------
* METHOD 1: COBOL INPUT AND OUTPUT PROCEDURES.
*-----------------------------------------------------------------
* 1. PRE-PROCESS:
* THE PROGRAM USES A SORT INPUT PROCEDURE TO DELETE RECORDS
* WITH A ZZZZZ OMIT FIELD BEFORE SORTING. THE OMIT FIELD IS
* IN COLUMNS 30-34.
* 2. SORT
* THE PROGRAM CALLS DFSORT TO SORT THE RECORDS IN DESCENDING
* ORDER. THE KEY IS IN COLUMNS 5-24.
* 3. POST-PROCESS:
* THE PROGRAM USES A SORT OUTPUT PROCEDURE TO WRITE ONE
* RECORD WITH EACH KEY AFTER SORTING.
*
* INPUT/OUTPUT: READS INDS AND WRITES OUTDS.
* DFSORT PASSES RECORDS TO THE PROCEDURES.
*-----------------------------------------------------------------
ID DIVISION.
PROGRAM-ID. CASE1.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT INDS ASSIGN TO INDS.
SELECT OUTDS ASSIGN TO OUTDS.
SELECT SORT-FILE ASSIGN TO SORTFILE.
DATA DIVISION.
FILE SECTION.
FD INDS RECORD CONTAINS 160 CHARACTERS
LABEL RECORD STANDARD BLOCK 27840
DATA RECORDS ARE INDS-RECORD.
01 INDS-RECORD.
05 FILLER PIC X(4).
05 INDS-KEY PIC X(20).
05 FILLER PIC X(5).
05 INDS-OMIT PIC X(5).
05 FILLER PIC X(126).
FD OUTDS RECORD CONTAINS 160 CHARACTERS
LABEL RECORD STANDARD BLOCK 27840
DATA RECORDS ARE OUTDS-RECORD.
01 OUTDS-RECORD.
05 FILLER PIC X(160).
SD SORT-FILE RECORD CONTAINS 160 CHARACTERS
DATA RECORD SORT-RECORD.
01 SORT-RECORD.
05 FILLER PIC X(4).
05 SORT-KEY PIC X(20).
05 FILLER PIC X(136).
WORKING-STORAGE SECTION.
01 FLAGS.
05 INDS-EOF PIC X VALUE SPACE.
88 SFLAG VALUE "Y".
05 TEMP-EOF PIC X VALUE SPACE.
88 TFLAG VALUE "Y".
01 PSTART PIC 9(1) VALUE 0.
01 SAVE-KEY PIC X(20).
01 TEMP-RECORD.
05 FILLER PIC X(4).
05 TEMP-KEY PIC X(20).
05 FILLER PIC X(136).
PROCEDURE DIVISION.
MASTER SECTION.
*-----------------------------------------------------------------
* CALL DFSORT TO SORT THE RECORDS IN DESCENDING ORDER.
*-----------------------------------------------------------------
SORT SORT-FILE
ON DESCENDING KEY SORT-KEY
INPUT PROCEDURE INPUT-PROC
OUTPUT PROCEDURE OUTPUT-PROC.
IF SORT-RETURN > 0
DISPLAY "SORT FAILED".
STOP RUN.
*-----------------------------------------------------------------
* SORT INPUT PROCEDURE:
* READ INDS.
* DELETE ALL RECORDS WITH A 'ZZZZZ' OMIT FIELD.
* SEND ALL OTHER RECORDS TO DFSORT FOR SORTING.
*-----------------------------------------------------------------
INPUT-PROC SECTION.
OPEN INPUT INDS
READ INDS AT END SET SFLAG TO TRUE
END-READ
PERFORM UNTIL SFLAG
IF INDS-OMIT NOT = "ZZZZZ"
RELEASE SORT-RECORD FROM INDS-RECORD
END-IF
READ INDS AT END SET SFLAG TO TRUE
END-READ
END-PERFORM.
CLOSE INDS.
*-----------------------------------------------------------------
* SORT OUTPUT PROCEDURE:
* RECEIVE RECORDS FROM DFSORT INTO TEMP.
* WRITE ONE RECORD WITH EACH KEY TO OUTDS.
*-----------------------------------------------------------------
OUTPUT-PROC SECTION.
OPEN OUTPUT OUTDS
RETURN SORT-FILE INTO TEMP-RECORD AT END SET TFLAG TO TRUE
END-RETURN
PERFORM UNTIL TFLAG
IF PSTART = 0
*-----------------------------------------------------------------
* FIRST RECORD - SAVE KEY AND WRITE RECORD TO OUTDS.
*-----------------------------------------------------------------
MOVE TEMP-KEY TO SAVE-KEY
WRITE OUTDS-RECORD FROM TEMP-RECORD
MOVE 1 TO PSTART
ELSE
IF TEMP-KEY NOT = SAVE-KEY
*-----------------------------------------------------------------
* RECORD HAS NEW KEY - SAVE KEY AND WRITE RECORD TO OUTDS.
*-----------------------------------------------------------------
MOVE TEMP-KEY TO SAVE-KEY
WRITE OUTDS-RECORD FROM TEMP-RECORD
END-IF
END-IF
RETURN SORT-FILE INTO TEMP-RECORD
AT END SET TFLAG TO TRUE
END-RETURN
END-PERFORM.
CLOSE OUTDS.
Application considerations 55
Application Considerations
DFSORT treats the INPUT PROCEDURE as an E15 user exit which must supply all the input records.
DFSORT calls the INPUT PROCEDURE once for each input record. The INPUT PROCEDURE reads the input
data set and uses RELEASE to pass each record with a non-zero OMIT field to DFSORT.
DFSORT sorts the records passed to it by the INPUT PROCEDURE as requested by the SORT statement
passed to it by the calling program.
DFSORT treats the OUTPUT PROCEDURE as an E35 user exit which must dispose of all the output records.
DFSORT calls the OUTPUT PROCEDURE once for each sorted record. The OUTPUT PROCEDURE uses
RETURN to obtain the records passed from DFSORT and writes one record with each key to the output
data set.
Performance
Since the COBOL program's INPUT and OUTPUT PROCEDUREs must, by definition, read and write the data
sets, NOFASTSRT is in effect for Method 1. FASTSRT is in effect for the other methods we will describe,
providing significant performance improvements.
DFSORT reads the input data set and deletes each record with a zero OMIT field as requested by the
OMIT statement.
DFSORT sorts the remaining records as requested by the SORT statement passed from the calling
program.
DFSORT writes one record with each key to the output data set as requested by the SUM statement.
Productivity
The use of DFSORT editing functions rather than COBOL code reduces considerably the effort required to
perform the application. Source code for pre-processing and post-processing is eliminated along with the
time to compile and debug the code.
In addition, future changes to the editing functions performed by the application can be made by simply
changing the control statements. Access to source code or recompiles are not necessary.
Control statements
SORTCNTL contains the OMIT and SUM statements used with the COBOL calling program.
//SORTCNTL DD *
OMIT COND=(30,5,CH,EQ,C'ZZZZZ')
SUM FIELDS=NONE
/*
*-----------------------------------------------------------------
* METHOD 2: COBOL MAIN PROGRAM.
*-----------------------------------------------------------------
* 1. PRE-PROCESS:
* A DFSORT OMIT CONTROL STATEMENT DELETES RECORDS WITH A
* 'ZZZZZ' OMIT FIELD BEFORE SORTING. THE OMIT FIELD IS IN
* IN COLUMNS 30-34.
* 2. SORT
* THE PROGRAM CALLS DFSORT TO SORT THE RECORDS IN DESCENDING
* ORDER. THE KEY IS IN COLUMNS 5-24.
* 3. POST-PROCESS:
* A DFSORT SUM CONTROL STATEMENT WRITES ONE RECORD WITH
* EACH KEY AFTER SORTING.
*
* INPUT/OUTPUT: DFSORT READS SORTIN AND WRITES SORTOUT.
*-----------------------------------------------------------------
ID DIVISION.
PROGRAM-ID. CASE2.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT SORTIN ASSIGN TO SORTIN.
SELECT SORTOUT ASSIGN TO SORTOUT.
SELECT SORT-FILE ASSIGN TO SORTFILE.
DATA DIVISION.
FILE SECTION.
FD SORTIN RECORD CONTAINS 160 CHARACTERS
LABEL RECORD STANDARD BLOCK 27840
DATA RECORDS ARE SORTIN-RECORD.
01 SORTIN-RECORD.
05 FILLER PIC X(160).
FD SORTOUT RECORD CONTAINS 160 CHARACTERS
LABEL RECORD STANDARD BLOCK 27840
DATA RECORDS ARE SORTOUT-RECORD.
01 SORTOUT-RECORD.
Application considerations 57
Application Considerations
Performance
Since Method 2 uses USING and GIVING rather than INPUT and OUTPUT PROCEDUREs, FASTSRT is in
effect. In addition, by eliminating the overhead related to passing each record between DFSORT and the
user exits, and enabling DFSORT to use its highly optimized editing code, Method 2 achieves significant
performance gains in CPU time, elapsed time, and EXCPs compared to Method 1. Figure 11 on page 58
shows a performance comparison between Method 1 and Method 2.
//SORTDIAG DD DUMMY
//DFSHOW DD SYSOUT=*
//DFSPARM DD *
MSGDDN=MYSHOW
/*
Control statements
SYSIN contains the OMIT, SORT and SUM statements used with the direct invocation of DFSORT.
//SYSIN DD *
OMIT COND=(30,5,CH,EQ,C'ZZZZZ')
SORT FIELDS=(5,20,CH,D)
SUM FIELDS=NONE
/*
Operation
The system calls DFSORT directly.
DFSORT reads the input data set and deletes each record with a zero OMIT field as requested by the
OMIT statement.
DFSORT sorts the remaining records as requested by the SORT control statement.
DFSORT writes one record with each key to the output data set as requested by the SUM statement.
Productivity
Elimination of all COBOL code reduces significantly the effort required to perform an application. You can
use control statements to duplicate complex code logic quickly and effectively. Source code for pre-
processing, sorting, and post-processing is eliminated along with the time to compile and debug the code.
In addition, future changes to the editing and sorting functions performed by the application can be made
by simply changing the control statements. Access to source code, and recompiles, are not necessary.
Performance
Performance for Method 3 is comparable to that for Method 2.
Application considerations 59
Application Considerations
Tuning activity is often started when a particular job is taking too long to complete or is using system
resources excessively. But if such a “problem job” does not exist at your site, where do you start to look
for ways to improve DFSORT's performance or balance its use of resources with other requirements?
The purpose of this chapter is to outline the actions available for those who want to tune the use of
DFSORT at their site. It also shows what information you need about DFSORT, where to find it, and what
methods are available for collecting the data, including use of the DFSORT installation exits ICEIEXIT and
ICETEXIT. The specific topics include:
• Where to find performance indicators
• An overview of DFSORT performance information
• Sources of DFSORT performance information
• Techniques for analyzing DFSORT performance data
• Balancing DFSORT requirements with system resources
• Understanding trade-offs in improving performance
JES Log
For batch jobs, JES writes a job log including messages issued and system accounting information.
Figure 13 on page 62 shows a sample JES2 log for a DFSORT job. The JES2 log produced at your site
will, of course, be different.
J E S 2 J O B L O G -- S Y S T E M E D S 3 -- N O D E S N J M A S 3
Messages
Some programs (like DFSORT) issue messages quantifying their use of certain system resources. This
is the simplest way of accessing such information. In the case of DFSORT, when a SORTDIAG DD
statement is present or installation option DIAGSIM=YES has been specified for your site, additional
messages useful for tuning are issued. An example of the DFSORT messages is shown in Figure 14 on
page 63. See z/OS DFSORT Messages, Codes and Diagnosis Guide for explanations of these
messages.
ICEIEXIT
The installation initialization exit (ICEIEXIT) allows you to examine certain installation and run-time
information for each DFSORT application. The ICEIEXIT routine could be used to collect the
information relevant to performance, and to write this information to a data set which can be
analyzed. Refer to “Using ICEIEXIT data ” on page 69 for information on using ICEIEXIT data to do a
moderate analysis of your DFSORT applications. Refer to “Installation exits ” on page 32 for a brief
overview of ICEIEXIT.
SMF
System management facilities (SMF) collects a variety of system and application-related information
into a number of different SMF records written by the system and various programs.
SMF type-30 (subtype 4) records, for instance, are written for each job step, and summarize the step's
consumption of major system resources, such as CPU time (broken down into these fields: TCB, SRB,
RCT, HPT, and IIP), elapsed time, EXCP counts, and device connect times. In addition, DFSORT can
write an SMF type-16 record, which summarizes the key statistics from a particular DFSORT run. By
running reports against SMF data, sites can use this information for workload analysis, planning, and
accounting purposes.
To extract performance information from SMF data, you can run an SMF report, use a data reduction
program, use the analysis and reporting features of DFSORT or its ICETOOL utility, or write a program
of your own. Since SMF data often contains accounting and other sensitive data, access might be
limited.
ICETEXIT
The installation termination exit (ICETEXIT) allows you to collect extensive performance-related
information about all DFSORT applications at a site. ICETEXIT provides comprehensive data for each
DFSORT application including the information contained in DFSORT's type-16 SMF record. Refer to
“Using ICETEXIT data ” on page 70 for information on using ICETEXIT data to do a moderate
analysis of your DFSORT applications. Refer to “Installation exits ” on page 32 for a brief overview of
ICETEXIT.
RMF
Resource Measurement Facility (RMF) measures system, address space, and workload activity. You
can use RMF to generate reports online or off-line, or as a real-time monitor. Most of these reports
concern performance-related statistics for processor, storage, I/O devices, and system data sets.
RMF's primary use is for system-level tuning, but it can also detail the performance characteristics of
subsystems and workloads. Many system programmers rely heavily on RMF reports for these analysis
activities. RMF writes a series of records into the SMF system data sets which can later be processed
by RMF, or other SMF analysis programs.
In terms of DFSORT performance analysis, RMF can be used to understand DFSORT's use of
resources, to examine the effects of running DFSORT applications, and to highlight contention for
resources. For example, RMF can help you identify situations where multiple work data sets are being
placed on the same disk device, by showing a high device utilization for that particular device. As with
SMF, access to RMF and its data might be restricted.
For a detailed cross reference of sources of performance indicators, refer to Table 7 on page 65.
ICExxxx
DFSORT messages (The ICE8xxI messages only appear if you have coded the SORTDIAG DD
statement or if the installation option DIAGSIM=YES has been specified for your site.). See z/OS
DFSORT Messages, Codes and Diagnosis Guide for explanations of the DFSORT messages.
JLOG
Job/JES log.
SMF16
SMF type-16 record (issued by DFSORT).
SMF30
SMF type-30 (subtype 4) record.
IEXIT
Information passed to the DFSORT ICEIEXIT routine.
TEXIT
Information passed to the DFSORT ICETEXIT routine (includes contents of the SMF type-16 record).
An individual source might not contain all of the information listed under Analysis.
DFSORT/ICETOOL
If you do not have access to a performance analysis and reporting product, you can use the DFSORT
editing functions in combination with the reporting capabilities of ICETOOL or OUTFIL to generate
performance data reports. The INCLUDE/OMIT, OUTFIL, and INREC/OUTREC control statements are
helpful in selecting the particular fields of the particular SMF, RMF, or other data records to be analyzed.
ICETOOL or OUTFIL can then be used to generate printable reports from the resulting raw data. See z/OS
DFSORT Application Programming Guide for more information about ICETOOL, OUTFIL, and editing
functions.
the level of analysis you are performing. See z/OS DFSORT Application Programming Guide and z/OS
DFSORT Installation and Customization for more information about the ICETOOL DEFAULTS operator.
Simple analysis
Generally, a few DFSORT applications at a site can be identified as being the largest and longest running,
or on a critical path. An application is said to be on a critical path if any delay in its completion would delay
finishing the workload. Tuning activities on the critical path can have a beneficial effect in reducing the
overall elapsed time of the batch workload. The performance of these applications can determine
whether you finish inside the batch window or overrun it, or whether you are using too many system
resources. The "80/20 rule of thumb" applies to DFSORT: often 80% of the system's resources used by
DFSORT are used by 20% of DFSORT applications.
Start your investigation by concentrating on these applications. Use a SORTDIAG DD statement (or make
sure the ICEMAC option DIAGSIM=YES has been specified for your site) to obtain all available messages.
DFSORT messages are a simple and effective source of information for individual applications. They give
you a picture of the processing and help you develop an action plan.
Performance-related information in the DFSORT messages includes:
• Amount of virtual storage available
• DFSORT technique used
• Total bytes and records sorted
• Number and distribution of EXCPs by data set
• Work data set space allocated and used
• Hiperspace, data space, or memory object storage used
• Availability of the DFSORT SVC
• Access methods and block sizes used
• Settings of various options (for example, EQUALS, VERIFY) that can affect performance
Moderate analysis
If you can take the time to do some analysis of all your DFSORT applications, you can pinpoint more
precisely where to tune your use of DFSORT to perform more efficiently. The easiest way to start is to use
information optionally provided by DFSORT. To do this, you can use data from SMF records or an ICEIEXIT
routine.
Type-30 records
SMF type-30 (subtype 4) records contain useful information on the resource consumption of a particular
job step. These include:
• CPU time (total, and broken down by field)
• Elapsed time
• EXCPs (total, and broken down by device)
• Device connect time, broken down by device
• Paging statistics
See z/OS MVS System Management Facilities (SMF) for more detailed information on the type-30 fields.
Type-16 records
DFSORT produces SMF type-16 records. They contain data on a particular DFSORT step. If you want to
produce DFSORT SMF records, DFSORT's supplied ICEMAC default of SMF=NO must be changed to
SMF=FULL or SMF=SHORT, or SMF=FULL or SMF=SHORT must be specified on an OPTION statement in
DFSPARM or in the extended parameter list at run-time. This must be done in addition to setting up the
appropriate SMFPRMxx member. The DFSORT SVC must be available for DFSORT to be able to write the
SMF records to an SMF data set. Refer to “Making the DFSORT SVC available” on page 20 for additional
information.
Some of the fields in the type-16 record containing performance-related data are:
• Type of application (sort, merge, or copy)
• DFSORT technique used
• Elapsed, TCB, and SRB time used
• Type and length of records being sorted
• Total bytes and records sorted
• Work data set allocation data
• Access methods and block sizes used
• Control field length
• Amount of Hiperspace, data space, and memory object storage used
• Number of input and output data set I/O calls
• Number of work data set EXCPs
• Number of intermediate merges performed
See z/OS DFSORT Installation and Customization for a complete list of the type-16 fields.
There are several reasons why it might be convenient to use the DFSORT type-16 records:
• You already have procedures to analyze this information.
• The scope of the information you are interested in is limited to the information available in the SMF
records.
• You do not wish to write and maintain an analysis routine.
Type-42 records
DFSMS/MVS produces SMF type-42 (subtype 6) records. They contain information on a particular disk
data set OPEN for a particular job, such as:
• Start Subchannels (SCCHs) (physical I/Os)
• Response time components (Connect, Disconnect, IOSQ and Pending times)
• Cache performance information
Both interval and CLOSE-time records are produced (unless interval recording is disabled).
See z/OS MVS System Management Facilities (SMF) for more detailed information on the type-42 subtype
6 fields.
Thorough analysis
To tune DFSORT thoroughly, you need extensive data about all DFSORT applications run by you or your
site, including elapsed time, CPU time, number of EXCPs, and types and amounts of virtual storage used.
JES logs and DFSORT messages can be used to analyze an isolated number of applications, but this is not
practical to apply to a large number of DFSORT jobs. In addition, these sources do not supply all of the
data needed to perform such a thorough analysis of DFSORT resource consumption.
In some cases, you only need a portion of this information for performance and tuning reasons.
The advantage of using an ICETEXIT routine is that all the information about each DFSORT application is
available to the routine. You do not need to use information from a number of other sources. How you
process the data depends on your requirements.
Examples of how you can use an ICETEXIT routine include:
• Collating information from SMF type-16 records with run-time and installation option values to identify
applications with options which degrade performance.
• Write an SMF record using your own format.
You can write the record to a private data set for subsequent processing.
You can process SMF records or private data set records using an appropriate data reduction program.
The example in Appendix A, “Sample ICETEXIT,” on page 75 provides a sample ICETEXIT assembler
routine. This installation-wide exit creates a user SMF record from the data passed by DFSORT.
The assembler source for an SVC which writes the user SMF record to SMF is also included. This enables
the ICETEXIT routine to run unauthorized but write SMF records using the SVC.
See z/OS DFSORT Installation and Customization for more information on how to write and install an
ICETEXIT routine,
the Channel Activity and I/O Queueing Activity reports) can help you to identify possible areas of device
contention.
For instance, if the report shows a high device activity rate and an unusually large pending time for a
particular device (or group of devices) containing DFSORT work data sets, the performance of the sort
application was most certainly adversely affected. If it turns out that the volumes selected for the work
data sets happened to, for example, also contain the tables for a very active data base, it might be
advantageous to make changes to the applications or to the I/O configuration that would prevent future
allocation of DFSORT work data sets to those particular devices.
The same idea holds true for DFSORT input and output data sets, and in general for any application on the
system; if there is a lot of contention for a few I/O devices, the data on those devices should be
distributed over more (less active) volumes to achieve an overall balance in I/O activity rates. Keep in
mind that for best DFSORT performance, the input and work data sets, as well as the work and output
data sets should be located on different volumes, or even different storage subsystems if possible.
Performance trade-offs
Table 8 on page 73 is a summary of the techniques described in this document for improving DFSORT
performance. In many cases (use of Blockset is a notable exception), an improvement in performance is
not free. The figure summarizes these potential trade-offs: increased paging, increased swapping,
increased CPU time, and changes needed to applications.
For example, to use this figure, assume you want to improve the elapsed time for a DFSORT application. A
number of techniques exist to choose from, ranging from ensuring that DFSORT uses Blockset to
modifying the way the application uses DFSORT. You can decide which technique, or combination of
techniques, is appropriate based on the effort required and the trade-off, if any, you are prepared to
accept.
Sample ICETEXIT
*---------------------------------------------------------------------*
* INITIALIZATION. *
*---------------------------------------------------------------------*
SPACE 1
SAVE (14,12),,ICETEXIT-&SYSDATE;-&SYSTIME;
LR R12,R15 EPA FOR BASE
USING ICETEXIT,R12
LR R11,R1 SAVE ICETPAR ADDR
USING ICETPAR,R11
LA R0,DSALTH LENGTH OF DSA TO GETMAIN
GETMAIN RC,LV=(0)
LTR R15,R15 DID WE GET STORAGE?
BNZ S01L990 NO..THEN GET OUT
ST R1,8(,R13) FORWARD CHAIN
ST R13,4(,R1) BACKWARD CHAIN
LR R13,R1 ->NEW SAVE AREA
USING DSA,R13
SPACE 1
*---------------------------------------------------------------------*
USING ICEGEN,R2
L R1,ICEGSTAT LENGTH OF GEN STATISTICS
LR R0,R10 TO ADDRESS
AR R9,R1 INCREMENT LENGTH/OFFSET
AR R10,R1 ->AFTER SMF RECORD
LR R3,R1 FROM LENGTH
MVCL R0,R2 APPEND DFSORT SMF RECORD
DROP R2
SPACE 1
*---------------------------------------------------------------------*
* APPEND THE OPTIONS STATISTICS. *
*---------------------------------------------------------------------*
SPACE 1
S01L130 DS 0H
ICM R2,B'1111',ICETOPTS -> OPTIONS STATISTICS
BZ S01L140 BRANCH IF NONE
STH R9,ICE81OOF SAVE OFFSET
USING ICEOPTS,R2
L R1,ICEOSTAT LENGTH OF OPTIONS STATISTICS
LR R0,R10 TO ADDRESS
AR R9,R1 INCREMENT LENGTH/OFFSET
AR R10,R1 ->AFTER SMF RECORD
LR R3,R1 FROM LENGTH
MVCL R0,R2 APPEND DFSORT SMF RECORD
DROP R2
SPACE 1
*---------------------------------------------------------------------*
* APPEND THE SORT/MERGE STATISTICS. *
*---------------------------------------------------------------------*
SPACE 1
S01L140 DS 0H
ICM R2,B'1111',ICETSMS -> SORT/MERGE STATISTICS
BZ S01L150 BRANCH IF NONE
STH R9,ICE81MOF SAVE OFFSET
USING ICESMS,R2
L R1,ICEFSTAT LENGTH OF SORT/MERGE STATISTICS
LR R0,R10 TO ADDRESS
AR R9,R1 INCREMENT LENGTH/OFFSET
AR R10,R1 ->AFTER SMF RECORD
LR R3,R1 FROM LENGTH
MVCL R0,R2 APPEND DFSORT SMF RECORD
DROP R2
SPACE 1
*---------------------------------------------------------------------*
* APPEND THE VIRTUAL STORAGE STATISTICS. *
*---------------------------------------------------------------------*
SPACE 1
S01L150 DS 0H
ICM R2,B'1111',ICETVIRT -> VIRTUAL STORAGE STATISTICS
BZ S01L160 BRANCH IF NONE
STH R9,ICE81VOF SAVE OFFSET
USING ICEVSTOR,R2
L R1,ICEVSTAT LENGTH OF VIRTUAL STORAGE STATISTICS
LR R0,R10 TO ADDRESS
AR R9,R1 INCREMENT LENGTH/OFFSET
AR R10,R1 ->AFTER SMF RECORD
LR R3,R1 FROM LENGTH
MVCL R0,R2 APPEND DFSORT SMF RECORD
DROP R2
SPACE 1
*---------------------------------------------------------------------*
* APPEND THE PHASE TIMING STATISTICS. *
*---------------------------------------------------------------------*
SPACE 1
S01L160 DS 0H
ICM R2,B'1111',ICETPTIM -> PHASE TIMING STATISTICS
BZ S01L170 BRANCH IF NONE
STH R9,ICE81TOF SAVE OFFSET
USING ICEPHAST,R2
L R1,ICEPTIME LENGTH OF PHASE TIMING STATISTICS
LR R0,R10 TO ADDRESS
AR R9,R1 INCREMENT LENGTH/OFFSET
AR R10,R1 ->AFTER SMF RECORD
LR R3,R1 FROM LENGTH
MVCL R0,R2 APPEND DFSORT SMF RECORD
DROP R2
SPACE 1
*---------------------------------------------------------------------*
* APPEND THE SORTIN STATISTICS. *
*---------------------------------------------------------------------*
SPACE 1
S01L170 DS 0H
Sample ICETEXIT 77
Sample ICETEXITICETEXIT
Sample ICETEXIT 79
Sample ICETEXITICETEXIT
SPACE 1
ICE81RCD DSECT
ICE81LEN DS H RECORD LENGTH
ICE81SEG DS H SEGMENT DESCRIPTOR
ICE81FLG DS XL1 SYSTEM INDICATOR
ICE81RTY DS AL1 RECORD TYPE = AL1(129)
ICE81TME DS XL4 TIME, IN HUNDREDTHS OF A SECOND
ICE81DTE DS PL4 DATE, IN FORM 00YYDDDF
ICE81SID DS CL4 SYSTEM IDENTIFICATION
SPACE 1
ICE81ROF DS H OFFSET TO SORT SMF RECORD (ICESMF )
ICE81SOF DS H OFFSET TO SMF STATS (ICESST )
ICE81GOF DS H OFFSET TO GENERAL STATS (ICEGEN )
ICE81OOF DS H OFFSET TO OPTION STATS (ICEOPTS )
ICE81MOF DS H OFFSET TO SORT/MERGE STAT (ICESMS )
ICE81VOF DS H OFFSET TO VIRT STOR. STAT (ICEVSTOR)
ICE81TOF DS H OFFSET TO TIMING STATS (ICEPHAST)
ICE81XOF DS H OFFSET TO SORTIN STATS (ICESRTIN)
ICE81ZOF DS H OFFSET TO SORTOUT STATS (ICESRTOT)
ICE81WOF DS H OFFSET TO SORTWK STATS (ICEWRKDS)
ICE81EOF DS H OFFSET TO SORTWK ENTRY (ICEWORK )
ICE81EEN DS H # OF ICEWORK ENTRIES
ICE81HOF DS H OFFSET TO HIPER STATS (ICEHIPER)
ICE81DOF DS H OFFSET TO DATASPACE STATS (ICEDATAS)
ICE81NOF DS H OFFSET TO MEM. OBJ. STATS (ICEMOSEC)
ICE81LTH EQU *-ICE81RCD LENGTH OF COMMON PORTION
ICESMF
ICEDTEX
PRINT NOGEN
CVT LIST=NO,DSECT=YES
IEESMCA
YREGS
END
Accessible publications for this product are offered through IBM Knowledge Center (www.ibm.com/
support/knowledgecenter/SSLTBW/welcome).
If you experience difficulty with the accessibility of any z/OS information, send a detailed email message
to mhvrcfs@us.ibm.com.
Accessibility features
Accessibility features help users who have physical disabilities such as restricted mobility or limited vision
use software products successfully. The accessibility features in z/OS can help users do the following
tasks:
• Run assistive technology such as screen readers and screen magnifier software.
• Operate specific or equivalent features by using the keyboard.
• Customize display attributes such as color, contrast, and font size.
Accessibility 83
84 z/OS: DFSORT Tuning Guide
Notices
This information was developed for products and services that are offered in the USA or elsewhere.
IBM may not offer the products, services, or features discussed in this document in other countries.
Consult your local IBM representative for information on the products and services currently available in
your area. Any reference to an IBM product, program, or service is not intended to state or imply that only
that IBM product, program, or service may be used. Any functionally equivalent product, program, or
service that does not infringe any IBM intellectual property right may be used instead. However, it is the
user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document.
The furnishing of this document does not grant you any license to these patents. You can send license
inquiries, in writing, to:
The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically
made to the information herein; these changes will be incorporated in new editions of the publication.
IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this
publication at any time without notice.
This information could include missing, incorrect, or broken hyperlinks. Hyperlinks are maintained in only
the HTML plug-in output for the Knowledge Centers. Use of hyperlinks in other output formats of this
information is at your own risk.
Any references in this information to non-IBM websites are provided for convenience only and do not in
any manner serve as an endorsement of those websites. The materials at those websites are not part of
the materials for this IBM product and use of those websites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.
Licensees of this program who wish to have information about it for the purpose of enabling: (i) the
exchange of information between independently created programs and other programs (including this
one) and (ii) the mutual use of the information which has been exchanged, should contact:
IBM Corporation
Site Counsel
2455 South Road
Such information may be available, subject to appropriate terms and conditions, including in some cases,
payment of a fee.
The licensed program described in this document and all licensed material available for it are provided by
IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any
equivalent agreement between us.
Any performance data contained herein was determined in a controlled environment. Therefore, the
results obtained in other operating environments may vary significantly. Some measurements may have
been made on development-level systems and there is no guarantee that these measurements will be the
same on generally available systems. Furthermore, some measurements may have been estimated
through extrapolation. Actual results may vary. Users of this document should verify the applicable data
for their specific environment.
Information concerning non-IBM products was obtained from the suppliers of those products, their
published announcements or other publicly available sources. IBM has not tested those products and
cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of
those products.
All statements regarding IBM's future direction or intent are subject to change or withdrawal without
notice, and represent goals and objectives only.
This information contains examples of data and reports used in daily business operations. To illustrate
them as completely as possible, the examples include the names of individuals, companies, brands, and
products. All of these names are fictitious and any similarity to the names and addresses used by an
actual business enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs
in any form without payment to IBM, for the purposes of developing, using, marketing or distributing
application programs conforming to the application programming interface for the operating platform for
which the sample programs are written. These examples have not been thoroughly tested under all
conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these
programs. The sample programs are provided "AS IS", without warranty of any kind. IBM shall not be
liable for any damages arising out of your use of the sample programs.
Applicability
These terms and conditions are in addition to any terms of use for the IBM website.
Personal use
You may reproduce these publications for your personal, noncommercial use provided that all proprietary
notices are preserved. You may not distribute, display or make derivative work of these publications, or
any portion thereof, without the express consent of IBM.
Commercial use
You may reproduce, distribute and display these publications solely within your enterprise provided that
all proprietary notices are preserved. You may not make derivative works of these publications, or
Rights
Except as expressly granted in this permission, no other permissions, licenses or rights are granted, either
express or implied, to the publications or any information, data, software or other intellectual property
contained therein.
IBM reserves the right to withdraw the permissions granted herein whenever, in its discretion, the use of
the publications is detrimental to its interest or, as determined by IBM, the above instructions are not
being properly followed.
You may not download, export or re-export this information except in full compliance with all applicable
laws and regulations, including all United States export laws and regulations.
IBM MAKES NO GUARANTEE ABOUT THE CONTENT OF THESE PUBLICATIONS. THE PUBLICATIONS ARE
PROVIDED "AS-IS" AND WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED,
INCLUDING BUT NOT LIMITED TO IMPLIED WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT,
AND FITNESS FOR A PARTICULAR PURPOSE.
Notices 87
Minimum supported hardware
The minimum supported hardware for z/OS releases identified in z/OS announcements can subsequently
change when service for particular servers or devices is withdrawn. Likewise, the levels of other software
products supported on a particular release of z/OS are subject to the service support lifecycle of those
products. Therefore, z/OS and its product publications (for example, panels, samples, messages, and
product documentation) can include references to hardware and software that is no longer supported.
• For information about software support lifecycle, see: IBM Lifecycle Support for z/OS (www.ibm.com/
software/support/systemsz/lifecycle)
• For information about currently-supported IBM hardware, contact your IBM representative.
Trademarks
IBM, the IBM logo, and ibm.com® are trademarks or registered trademarks of International Business
Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked
terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these
symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information
was published. Such trademarks may also be registered or common law trademarks in other countries. A
current list of IBM trademarks is available on the Web at Copyright and Trademark information
(www.ibm.com/legal/copytrade.shtml).
UNIX is a registered trademark of The Open Group in the United States and other countries.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.
Other company, product, and service names may be trademarks or service marks of others.
89
data sets (continued) E35 exit 52, 54–56
creating with OUTFIL 6 elapsed time
disk 14 definition 4
dynamic allocation 11 ESCON channels 14
input and output 45 sources of data 64
placement 71 trade-offs 2, 73, 74
tape 14 Environment Installation Modules 26
temporary 16 environmental considerations 13
VIO 72 ESCON channels
work 14 elapsed time performance 14
data sources summary 64 virtual storage 42
data space sorting 6 EXCPs, execute channel program (EXCP) commands
Data Space Sorting 23 performance indicator 4
defaults sources of data 64
listing with ICETOOL 26
DEFAULTS operator (ICETOOL)
examples 26
F
listing installation defaults 26 FASTSRT 51, 52, 56, 58, 59
definitions feedback xv
Blockset technique 5 file size 12, 41
cache fast write (CFW) 9 FTP Site 1
channel usage 4
compression 11
CPU time 3 G
data space sorting 6
general information, tuning 1
device connect time 4
guidelines for virtual storage 43, 44
dynamic allocation 11
dynamic storage adjustment (DSA) 9
elapsed time 4 H
EXCPs, execute channel program (EXCP) commands 4
hipersorting 6 hierarchy, storage 13
HPT (Hiperspace Processing Time) 3 high speed buffer 64
I/O activity 4 hipersorting 6
ICEGENER 10 Hipersorting
IIP (I/O Interrupt Processing) 3 blockset 38
memory object sorting 6 Hiperspace Processing Time (HPT) 3
OUTFIL 6 hiperspace sorting 6
RCT (Region Control Task) 3 Hiperspace storage 2
SRB (Service Request Block) 3 HPT 3
system determined block size (SDB) 11
system paging activity 2 I
TCB (Task Control Block) 3
device connect time 4 I/O activity
devices performance indicator 4
3390 39 trade-offs 2, 73, 74
DFSORT performance 47 I/O Interrupt Processing (IIP) 3
DFSORT FTP Site 1 ICEGENER
DFSORT Home Page 1 benefits 20
DFSORT requirements comparison with IEBGENER 10
balancing with system resources 71 general information 10
DFSPARM 56, 59 ICEIEXIT 32, 63, 64, 69
disk ICEMAC 25
cache 64 ICEMAC options 29
general information 14 ICETEXIT 33, 64, 70, 75
trade-offs 2 ICETOOL 26
utilization 4 ICETPEX 7, 12, 15
work data sets 14 IDCAMS BLDINDEX
DSA 9 general information 12
dynamic allocation of work data sets 11 IEFUSI 26
dynamic storage adjustment (DSA) 9 IGZSRTCD 51, 56
IIP 3
INCLUDE control statement 56
E
input and output data sets
E15 exit 52, 54–56 block sizes 45
90
input and output data sets (continued) O
DFSORT performance 45
performance 46 OMIT control statement 56, 57, 59
INREC control statement 56 operation
installation considerations 19 cache fast write (CFW) 9
installation defaults dynamic storage adjustment (DSA) 9
changing 25 memory object sorting 7
listing with ICETOOL 26 options
using ICEIEXIT 32 ICEMAC 26
using ICEMAC 25 installation 26
installation exits run-time 48
advantages 32 OUTFIL 6
ICEIEXIT 63 OUTFIL control statement 56
ICETEXIT 64 OUTREC control statement 56
IEFUSI 26
tuning purposes 32
installation modules
P
Environment 26 paging
listing 26 system 3
Time-of-Day 26 trade-offs 2
installation options 26 Peerage/Vale technique 5
installing DFSORT 19 performance
intermediate merging 41 analyzing data 67
Internet 1 analyzing using ICETEXIT 69
invoking DFSORT 51 analyzing using ICETEXIT data 70
analyzing with SMF data 68
J gathering information 67
main storage 16
JES log 62, 64 moderate analysis 68
run-time options 48
simple analysis 68
K site-wide options 26
keyboard thorough analysis 70
navigation 81 trade-offs 72
PF keys 81 performance indicators
shortcut keys 81 channel usage 4
CPU time 3
device connect time 4
L disk utilization 4
elapsed time 4
limitations
EXCPs, execute channel program (EXCP) commands 4
virtual storage 43
HPT (Hiperspace Processing Time) 3
listing installation defaults 26
I/O activity 4
IIP (I/O Interrupt Processing) 3
M JES log 62
RCT (Region Control Task) 3
main storage RMF (Resource Measurement Facility) 64
environmental considerations 16 SMF (System Management Facilities) 64
minimum 16 SRB (Service Request Block) 3
options that tailor 26 system paging activity 3
memory object sorting 6, 7, 23, 35 TCB (Task Control Block) 3
merging where to find 61
intermediate 42 PL/I 51, 52
messages 5, 62, 64, 68 placement of data sets 71
processor cache 13
N processor utilization 3
program residency 19, 64
navigation
keyboard 81 R
NOFASTSRT 51, 52, 54, 56
RCT 3
recommendations
data space sorting 23
91
recommendations (continued) tape, general information 15
Hipersorting 23 Task Control Block (TCB) 3
Memory Object Sorting 23 TCB 3
storage 21 techniques for understanding use
Region Control Task (RCT) 3 moderate analysis 68
reports simple analysis 68
ICETOOL 67 thorough analysis 70
OUTFIL 6 Time-of-Day Installation Modules 26
RMF 64, 71 trade-offs
SMF 64 CPU time 73, 74
required knowledge xi disk utilization 2
residency 64 elapsed time 2, 73, 74
Resource Measurement Facility (RMF) 3, 64, 71 Hipersorting 73, 74
RMF 3 I/O activity 2, 73, 74
run-time considerations 35 processor load 2
run-time options 48 system paging activity 2
virtual storage 73, 74
work data sets 73, 74
S tuning
SAS, using with DFSORT 12 during installation 19
SDB 11 examples 2
sending to IBM general information 1
reader comments xv importance 1
Service Request Block (SRB) 3 performance indicators 3
shortcut keys 81 purpose 2
SKIPREC option 56
SMF U
type 16 69
type-30 records 68 user interface
SMS 16 ISPF 81
SORT control statement 59 TSO/E 81
SORTCNTL 56, 57 User SVC 75
SORTDIAG 5, 62, 68 utilization
sources of data summary 64 disk 4
SRB 3 processor 3
STOPAFT option 56
storage
central 14
V
control cache 14 VIO (virtual input output)
hierarchy 13 analyzing use of data sets 72
main 16 DFSORT performance 48, 73, 74
options 21 virtual storage
system-managed 16 analyzing use 72
understanding options 21 data set size 41
using efficiently 41 DFSORT performance 41
virtual 15 environmental considerations 15
Storage Management Subsystem (SMS) 16 ESCON channels 42
SUM control statement 56, 57, 59 guidelines 43
SVC memory object sorting 15
DFSORT SVC 20, 64, 68, 75 sorting with data space 44
option 75 sources of data 64
User SVC 75 trade-offs 73, 74
SYSIN 59 VSAM, options that affect performance 26
system determined block size (SDB) 11
System Management Facilities (SMF) 20, 64, 68, 70, 75
system resources W
balancing with DFSORT requirements 71
work data sets
system-managed storage 16
cache fast write (CFW) 9
disk 14
T dynamic allocation 11
JCL 11
tape management system 12 options that affect allocation 26
tape work data sets 14 options that influence allocation 26
92
work data sets (continued)
sources of data 64
space trade-offs 73, 74
tape 15
World Wide Web 1
93
94
IBM®
SC23-6882-30