Cost Based Optimizer - Part 1 of 2

Cost Based Optimizer – 1 of 2 Hotsos Enterprises, Ltd. Grapevine, Texas Oracle. Performance. Now. [email_address]

Agenda Cost Based Optimizer and its impact in performance Data Points Collected by the Statistics Gathering Process The 10053 Trace File Case Studies

Cost Based Optimizer (CBO) The CBO in reality is a complex decision making software Use several Database Initialization Parameters These are listed in the 10053 trace file Uses several session level initialization parameter These are parameters at the session level that override the database initialization parameters Uses statistics about the objects These statistics will be discussed later Hints to the optimizer Uses Statistics about the system (CPU, Disk etc) Use this information and makes decisions on the “best way” to generate an execution plan

CBO will be part of your life if you keep working with Oracle. The cost-based query optimizer (CBO)… Uses data from a variety of sources Estimates the costs of several execution plans Chooses the plan it estimates to be the least expensive Characteristics Adapts to changing circumstances Frustrating if you don’t know what it considers as input Works great if you know how to use it But produces very poor results if you lie to it The only query optimizer supported by Oracle Corporation from release 10 onward

The cost-based query optimizer chooses the plan that it computes as having the lowest estimated cost. Don’t assume the following are identical CBO’s estimated cost of an execution plan The actual cost of an execution plan CBO’s cost estimate can be imperfect Are your CBO inputs perfect? CBO isn’t perfect, but by 9.2 it’s almost always good enough Without properly collected statistics, the CBO will use RBO if no statistics exist on any object in the statement use default statistics if statistics exist for a single object in the statement but not others use dynamic sampling to generate statistics (based on parameter setting and Oracle version)

Execution plan changes can result in profoundly different application performance. Table size change Device latency change Execution plan change Type C performance changes are the most profound size change performance change performance change performance change

Recap The CBO is a complex piece of software It uses several data points to calculate the cost of the execution plan and will choose the plan with the lowest cost It is dynamic and will adapt to changing data better than the Cost Based Optimizer A good understanding of the Cost Based Optimizer is imperative in understanding the rationale behind some of the choices

Data Points Collected by the Cost Based Optimizer

Data Points Collected by the Cost Based Optimizer Table Statistics Column Statistics Index Statistics System Statistics

Table Statistics Statistics collected for tables appear in the data dictionary views *_TABLES Number of rows ( NUM_ROWS ) Number of data blocks below the high water mark ( BLOCKS ) Number of data blocks allocated to the table that have never been used ( EMPTY_BLOCKS ) Average available free space in each data block in bytes ( AVG_SPACE ) Number of chained rows ( CHAIN_CNT ) Average row length, including the row's overhead, in bytes ( AVG_ROW_LEN )

Column Statistics Statistics collected for columns appear in the data dictionary views *_TAB_COLUMNS and *_TAB_COL_STATISTICS. Number of distinct column values ( NUM_DISTINCT ) Lowest value in a column ( LOW_VALUE ) Highest value in a column ( HIGH_VALUE ) Selectivity estimate for column ( DENSITY ) Number of null values in a column ( NUM_NULLS ) Number of histogram buckets ( NUM_BUCKETS ) Average column length ( AVG_COL_LEN ) Existence/type of histogram ( HISTOGRAM )

Index Statistics Statistics collected for indexes appear in the data dictionary views *_INDEXES . Depth of the index from its root block to its leaf blocks ( BLEVEL ) Number of leaf blocks ( LEAF_BLOCKS ) Number of distinct indexed values ( DISTINCT_KEYS ) Average number of leaf blocks in which each distinct value in the index appears ( AVG_LEAF_BLOCKS_PER_KEY ) Average number of data blocks in the table that are pointed to by a distinct value in the index ( AVG_DATA_BLOCKS_PER_KEY ) Clustering factor (how well ordered the rows are with respect to the indexed values) ( CLUSTERING_FACTOR )

System Statistics Statistic Description sreadtim average time (ms) to complete a single block read mreadtim average time (ms) to complete a multi-block read cpuspeed average CPU cycles per second (in millions) cpuspeednw estimate of CPU cycles (v10) ioseektim estimate of average read seek time (v10) iotfrspeed estimate of transfer speed of I/O system (v10) mbrc average multiblock read count (in blocks) maxthr maximum I/O system throughput (in bytes/second) slavethr average slave I/O throughput (in bytes/second)

DBMS_STATS DBMS_STATS are used to collect the above described data points There are several debates on what the % of data collected should be What the options to the DBMS_STATS should be on Our perspective It should be dependent on the best plans that can be generated for all the SQL being executed in the system The frequency and the % of stats being gathered itself should not become a malignant load on the system

Recap There are several data points that are used by the optimizer They are collected using DBMS_STATS All these data points help the optimizer in deciding to choose a execution plan The collection of statistics should be a function of the quality of the plans that can be generated and the special cases that will drive higher frequency and % estimate of statistics collection

Example of Data Points Gathered by the Optimizer

10053 Trace File The 10053 Trace File is the dump of the analysis done by the Cost Based Optimizer in figuring out the execution plan It is unlike the 10046 trace file which has a specific syntax and is usually the same across version barring a few exceptions (Millsap, Optimizing Oracle Performance) It should be used as the last resort for trouble shooting Usually the difference in the rows estimated by the Cost Based Optimizer and the Actual Rows from the execution plan is good enough to see where the problem is

See Webinar 1 – SQL Tuning with Trace Data and DBMS_XPLAN

10053 Trace File Structure Peeked Values of the Bind variables in the SQL Statement Parameters used by the Optimizer Base Table Statistics Table Stats Column Stats Index Stats Single Table Access Path Join Order Determination (Tries Several Combinations and determines the type of the join and the order of the join) Query Itself

Script to Generate 10053 Trace File alter session set max_dump_file_size = unlimited; alter session set tracefile_identifier = ‘Query1_10053’; alter session set events '10053 trace name context forever, level 1'; Explain plan for select * from dba_objects where owner=‘SCOTT’; alter session set events ’10053 trace name context off’; exit;

Recap The 10053 Trace File is a good tool to understand the reasoning and the data behind the optimizer Should usually be used a last resort Produce large amount of data due to the number of combinations and data points involved in making a decision Usually the row difference in the execution plan and the explain plan should be more than enough to understand why the optimizer is making a certain choice of row source operation

Index Scan versus Full Table Scan’s

Setup Create a table which is a copy of dba_objects create table myobj as select * from dba_objects; We will create indexes on owner and object_type We will now gather default statistics and run some queries We will create indexes on owner and object_id We will then study some simple queries on this table

Should I do a Full Table Scan or Use an Index? Here is a simple query Full Table Scan select object_type, count(1) from myobj where object_id between 10 and 11558 group by object_type; Index Range Scan select object_type, count(1) from myobj where object_id between 10 and 11557 group by object_type;

Data Points Full Table Scan Cost 180 Logical I/O’s 772 Index Scan Cost 179 Logical I/O’s 166

Shouldn’t Oracle minimize the number of Logical I/O’s? Yes Yes, it is probably the right thing to do No Because, it is probably not the right thing The answer is it depends on several things the most important being db_file_multiblock_read_count In our experiment it was set to 16 At 16, the cost or the time taken to execute the query by Oracle will be faster if it did a full table scan using the multi block read count than if it did a sequential read of the index and the table You can set different values for db_file_multiblock_read_count and see you can see the cost calculations and the plans change

Now let us set db_file_multiblock_read_count to 8 Full Table Scan select object_type, count(1) from myobj where object_id between 10 and 14301 group by object_type; Index Range Scan select object_type, count(1) from myobj where object_id between 10 and 14300 group by object_type;

Let us set it to 8 and see what happens Full Table Scan Cost 221 Logical I/O’s 772 Index Scan Cost 220 Logical I/O’s 211

So there are a lot of moving parts!!! Yes, there are. How do you manage that Keep your optimizer environment as stable as possible The only thing that should be changing are the data points collected by the DBMS_STATS gathering process Keep a copy of your old statistics Just in case you may have to revert back to old stats to get back to the good ol’ plan

Cardinality/Selectivity Cardinality The predicted number of rows generated by a row source operation That is what you see in the explain plan Selectivity Cardinality is calculated by estimating the selectivity which is the expected fraction of rows that will pass the predicate test Is very sensitive to number of distinct values Determines join order choice based on cardinality and hence selectivity

Simple Selectivity Calculate Range for the Query (Rng) Obtained from Query Calculate the difference between column high value and column low value (hival-loval) Obtained from dba_tab_columns Divide Rng and (hival-loval) Add N/num_distinct where N is the range bound count and num_distinct is the number of distinct values for this column which is obtained from dba_tab_columns

Sample Selectivity Calculation select object_type, count(1) from myobj where object_id between 10 and 11558 group by object_type; Range is 11558-10 = 11548 Column High Value – 69892 Column Low Value – 14 N = 2 Number of Distinct Values – 55618 Note the optimizer see that 10 as a lower bound value does not make sense and upgrades it to 14 Select (11548-14)/(69892-14)+(2/55618) * 55618 from dual gives cardinality which is 9190 and let us look at the explain plan cardinality

Cost What do we mean by the cost of the query? Oracle Performance Guide Cost = (Number of Single Block Reads * single read time + number of multi block reads * multi block read time + CPU Cycles / CPU Cycles / second ) /Time to read a single block Dimensional Analysis Show Cost is just a number Although it is the optimizer guess at how long the query will take to execute

Selectivity and Cardinality Impact cost and join strategies Selectivity and Cardinality impact cost and join strategies as we saw in the example You can predict the cost and behavior of your queries as the data volume increases The number of distinct values is very important and determines the selectivity and hence cardinality and hence access strategies and hence costs The number of distinct values from non-unique columns on which predicates are written and joined can severely be impacted the quality of the statistics being collected

So how can you know if a plan is good? Look at the cardinality of the explain plan Look at the rows actually obtained from the execution plan (from the trace file or the row source execution statistics) See how closely they match If there is a wide variance, study if it is impacting your query performance and determine what additional statistics need to be gathered

How can you predict the nature of plans due to data growth? Use the cardinality hint to study the performance of the query and its behaviour

10053_SCOTT.sql alter session set max_dump_file_size = unlimited; alter session set tracefile_identifier = 'MYOBJ_SCOTT'; alter session set events '10053 trace name context forever, level 1'; explain plan for select object_type,count(*) from myobj where owner='SCOTT' group by object_type; alter session set events '10053 trace name context off'; exit;

10053_sys.sql alter session set max_dump_file_size = unlimited; alter session set tracefile_identifier = 'SYS_OBJ'; alter session set events '10053 trace name context forever, level 1'; explain plan for select object_type,count(*) from myobj where owner='SYS' group by object_type; alter session set events '10053 trace name context off'; exit;

Col_stats.sql SELECT table_name "Table Name", column_name "Column Name", num_distinct "Number of Distinct Values", low_value "Low Value", high_value "High Value", density "Density", num_nulls "Number of Nulls", num_buckets "Number of Buckets", avg_col_len "Average Column Length", histogram "Histogram" FROM user_tab_columns WHERE table_name='MYOBJ';

Gather_table_stats.sql begin dbms_stats.gather_table_stats(USER,'MYOBJ'); end;

Index_stats.sql SELECT table_name "Table Name", index_name "Index Name", blevel "Blevel", leaf_blocks "Leaf Blocks", distinct_keys "Distinct Keys", AVG_LEAF_BLOCKS_PER_KEY "Avg Leaf Blocks per Key", AVG_DATA_BLOCKS_PER_KEY "Avg Data Blocks Per Key", CLUSTERING_FACTOR "Clustering Factor" FROM user_indexes WHERE table_name='MYOBJ';

Join_strategy_choice.sql select /*+ cardinality (e 6059) cardinality (d 200) */ e.empno,e.ename,d.dname,d.loc from emp e, dept d where e.deptno=d.deptno; select /*+ cardinality (e 6060) cardinality (d 200) */ e.empno,e.ename,d.dname,d.loc from emp e, dept d where e.deptno=d.deptno;

Raw_to_num.sql select column_name, raw_to_num(low_value) low_val, raw_to_num(high_value) high_val from user_tab_columns where table_name = 'MYOBJ' and column_name = 'OBJECT_ID'

Tab_stats.sql select table_name "Table Name", num_rows "No. of Rows", blocks "Blocks", empty_blocks "Empty Blocks", avg_space "Average Space", chain_cnt "Chain Count", avg_row_len "Average Row Length", sample_size "Sampe Size" from user_tables where table_name='MYOBJ';

Cost Based Optimizer - Part 1 of 2

More Related Content

Cost Based Optimizer - Part 1 of 2

Editor's Notes