This document provides an overview of upgrading to Oracle Database 19c and ensuring performance stability after the upgrade. It discusses gathering statistics before the upgrade to speed up the process, using AutoUpgrade for upgrades, and various testing tools like AWR Diff Reports and SQL Performance Analyzer to check for performance regressions after the upgrade. Maintaining good statistics and thoroughly testing upgrades are emphasized as best practices for a successful upgrade.
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Report
Share
1 of 77
Download to read offline
More Related Content
Performance Stability, Tips and Tricks and Underscores
1. Upgrade to Oracle Database 19c
Performance Stability, Tips and Tricks and Underscores
Mike Dietrich
Master Product Manager
Database Upgrade and Migration
Roy Swonger
Vice President
Database Upgrade, Utilities and Patching
Virtual Classroom Series
6. Your key to successful Database Upgrades
1 2 3 4
Download and install
Oracle 19c
Download and install
newest RU
Download and use
newest AutoUpgrade
eDelivery MOS: 2521164.1 MOS: 2485457.1
SPM, STA, RAT for Performance
Stability
7. Upgrade 19c | Speed it up
• Check when dictionary stats have been gathered the last time
• Refresh stats a day before the upgrade
column OPERATION format a40
set linesize 200
select to_char(max(END_TIME),'DD-MON-YY hh24:mi') LATEST, OPERATION
from DBA_OPTSTAT_OPERATIONS
where OPERATION in ('gather_dictionary_stats', 'gather_fixed_objects_stats')
group by operation;
8. Upgrade 19c | Make it run faster …
• Stats not gathered on several clustered indexes (bug 25286819)
• But wait … autoupgrade.jar does this for you already!
exec dbms_stats.gather_schema_stats('SYS');
exec dbms_stats.gather_index_stats('SYS','I_OBJ#');
exec dbms_stats.gather_index_stats('SYS','I_FILE#_BLOCK#');
exec dbms_stats.gather_index_stats('SYS','I_TS#');
exec dbms_stats.gather_index_stats('SYS','I_USER#');
exec dbms_stats.gather_index_stats('SYS','I_TOID_VERSION#');
exec dbms_stats.gather_index_stats('SYS','I_MLOG#');
exec dbms_stats.gather_index_stats('SYS','I_RG#');
9. Upgrade 19c | Effect of having good stats
• The larger the dictionary, the bigger the effect
• This example has been done with one of the tiny Hands-On Lab databases
Duration Reduction
No dictionary and fixed objects stats 15 min 55 sec
Gathered dictionary and fixed objects stats 14 min 10 sec 11 %
Gathered schema and cluster index stats 13 min 41 sec 3.4 % to previous
Total downtime saved 2 min 14 sec 14 % overall
10. Effect of Having Good Statistics | EBS
• Upgrade duration for Oracle E-Business Suite
Duration Reduction
No dictionary and fixed objects stats 10 hrs 56 min 52 sec
Gathered dictionary and fixed objects stats 52 min 42 sec 93 %
Gathered schema and cluster index stats 52 min 25 sec 0.5 % to previous
Total downtime saved 10 hrs 4 min 14 sec 93.5 % overall
11. Effect of Having Good Statistics
• Stale/no stats: 9:59:23.87 • Good stats: 0:00:02.33
ID OPERATION OPTIONS OBJECT_NAME
----- -------------------- -------------------- -----------
0 UPDATE STATEMENT
1 UPDATE DEPENDENCY$
2 FILTER
3 TABLE ACCESS FULL DEPENDENCY$
4 INDEX FULL SCAN I_OBJ2
5 INDEX FULL SCAN I_OBJ2
6 TABLE ACCESS BY INDEX ROWID BATCHED OBJ$
7 INDEX RANGE SCAN I_OBJ1
8 TABLE ACCESS BY INDEX ROWID BATCH OBJ$
9 INDEX RANGE SCAN I_OBJ1
ID OPERATION OPTIONS OBJECT_NAME
----- -------------------- -------------------- --------------
0 UPDATE STATEMENT
1 UPDATE DEPENDENCY$
2 FILTER
3 TABLE ACCESS FULL DEPENDENCY$
4 INDEX RANGE SCAN I_OBJ1
5 INDEX RANGE SCAN I_OBJ1
6 TABLE ACCESS BY INDEX ROWID BATCHED OBJ$
7 INDEX RANGE SCAN I_OBJ1
8 TABLE ACCESS BY INDEX ROWID BATCH OBJ$
9 INDEX RANGE SCAN I_OBJ1
16. Parameter Recommendations
General guidelines:
• The fewer parameters you have in your spfile, the better
• Remove outdated parameters, underscores, events ...
• Unless directed by applications (e.g. E-Business Suite, Siebel, SAP etc.) to set them
• For EBS: Interoperability notes
17. Things to do right after upgrade
Configure statistics history retention period
• Check space usage:
• Check retention
• Default: 31 days
• Adjust setting
• Example: 10 days
SQL> select SPACE_USAGE_KBYTES/1024 MB from
V$SYSAUX_OCCUPANTS where
OCCUPANT_NAME='SM/OPTSTAT';
SQL> select
DBMS_STATS.GET_STATS_HISTORY_RETENTION from DUAL;
SQL> exec
DBMS_STATS.ALTER_STATS_HISTORY_RETENTION(10);
18. Help me - I have an upgrade problem …
In >95%of all cases
PhotobynikkomacaspaconUnsplash
“ ”
an "upgrade problem" in reality is a performance issue
after upgrade, or not a database upgrade topic.
There is exactly one way to mitigate the risk:
TESTING!
19. The entire upgrade project of our 300 Oracle
databases at La Mobilier is running very well.
When we catch issues, we fix them before
going live.
Paolo Kreth
Group Manager Databases
La Mobiliere
“
”
20. Testing | Typical Mistakes
• Only 10% of real data used
• Data sets artificially created
• Tests done on a laptop
• No testing tools used
• Data from 6 months ago
• No statistics refreshed
• Testing?? Waste of time!
• Real experts fix it after go-live …
PhotobyElijahO'DonnellonUnsplash
21. Best Practice | COMPATIBLE vs Optimizer
• COMPATIBLE and OPTIMIZER_FEATURES_ENABLE
• Fully independent from each other
• Set COMPATIBLE to the default of the release, e.g. 19.0.0
• Change OPTIMIZER_FEATURES_ENABLE only when you have to
• Avoid it if possible
• This is not a Swiss Army knife!
• You will turn off a lot of great features
22. Testing | Completeness versus Costs
Costs
Completeness
low high
0%100%
80% - 90%
Costslow high
Completeness
0%100%
23. Testing Tools | Workflow
Collect execution
plans before upgrade
Compare AWR
Snapshots
Verify them with
SQL Performance
Analyzer
Regressed plans?
SQL Plan
Management
SQL Tuning Advisor
Verify functionality
and performance with
Database Replay
24. Testing Tools | Collect Plans on Production
STS
Capturing from cursor cache and AWR
into an STS does not require an extra
license for EE anymore:
Find it under: "SQL Tuning Sets" at:
https://docs.oracle.com/database/121/DBLIC/editions.htm#DBLIC109
25. Testing Tools | Comparison
Collect execution
plans before upgrade
Compare AWR
Snapshots
Verify them with
SQL Performance
Analyzer
Regressed plans?
SQL Plan
Management
SQL Tuning Advisor
Verify functionality
and performance with
Database Replay
30. Testing Tools | SQL Plan Management
Collect execution
plans before upgrade
Compare AWR
Snapshots
Verify them with
SQL Performance
Analyzer
Regressed plans?
SQL Plan
Management
Verify functionality
and performance with
Database Replay
31. Testing Tools | Situation after change
• Challenging to "freeze" execution plans
Parse Execute
Plan acceptable
Plan possibly
not acceptable
Conditions change:
New statistics, parameter change, upgrade
GB
HJ
...HJ
GB
NL
...NL
SQL
SQL
ExecuteParse
32. SQL Plan Management | Mechanism
• Phase 1 – Baseline Capture
• Set OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=TRUE
• If OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=TRUE then all found plans will be put into the baseline
Parse Execute
SQL
GB
HJ
...HJ
SQL Plan acceptable
2X
GB
HJ
...HJ
Plan Baseline
SQL MANAGEMENT BASE
Residing in SYSAUX TS.
Will occupy max. 10% of SYSAUX.
Weekly job will delete plans
not used since 53 weeks [default].
Before Oracle 12c a massive number of hints get stored –
since Oracle 12c entire plans will be kept
SQL Profiles
Initial plan
will be accepted
if repeated and
will be added to the
SQL Plan Baseline
Plan History
33. SQL Plan Management | Mechanism
• Phase 2 – Selection
• Same statement parsed again but a different plan will be created
GB
HJ
...HJ
Plan Baseline
Plan History
Parse Execute
SQL
GB
NL
...NL
Plan acceptable
• OPTIMIZER_USE_SQL_PLAN_BASELINES=TRUE
• OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=FALSE
• Both switched to their defaults
• Annotation:
If OCSPB=TRUE then this plan would have been added to
the baseline
New plan will be added
to the Plan History but
it won‘t be used unless
it has been verified
GB
NL
...NL
34. SQL Plan Management | Mechanism
• Phase 3 – Evolution
• Since Oracle 12.1.0.2:
GB
HJ
...HJ
Plan Baseline
Plan History
GB
NL
...NL
GB
NL
HJHJ
GB
HJ
...HJ
Plan Baseline
Plan History
GB
NL
...NL
GB
NL
HJHJ
Equal or better plans
can be added to the
SQL Plan Baseline
Optimizer
Worse plans
will be kept
in the Plan
History and
will get purged
New SYS_AUTO_SPM_EVOLVE_TASK
job as part of the Automatic SQL Tuning Task
Report: DBMS_SPM.REPORT_AUTO_EVOLVE_TASK
Manual: DBMS_SPM.CREATE_EVOLVE_TASK
35. SQL> select PARAMETER_NAME, PARAMETER_VALUE
from DBA_SQL_MANAGEMENT_CONFIG;
SQL Plan Management | Settings
• Configure SQL Plan Management (SPM)
• Check settings:
• Change retention:
• Default: 53 weeks
• Change space consumption:
• Default: 10% of SYSAUX
• Plans will be stored in a LOB
• Sources to load plans from:
SQL> exec DBMS_SPM.LOAD_PLANS_FROM_AWR (begin_snap=>29, end_snap=>69);
SQL> exec
DBMS_SPM.CONFIGURE('plan_retention_weeks',5);
SQL> exec
DBMS_SPM.CONFIGURE('space_budget_percent',5);
36. SQL Plan Management | Plan Stability
GB
HJ
...HJ
Plan Baseline
Plan History
STS
STSExport/Import for Migrations
Upgrade
Plans will be takenfrom STS and
addedto the SQL Plan Baseline
DBMS_SPM.LOAD_PLANS_FROM_SQLSET
37. Testing Tools | SQL Tuning Advisor
Collect execution
plans before upgrade
Compare AWR
Snapshots
Verify them with
SQL Performance
Analyzer
Regressed plans?
SQL Plan
Management
SQL Tuning Advisor
Verify functionality
and performance with
Database Replay
38. SQL Tuning Advisor | SQL Profiles
• SQL Profiles provide more information to the optimizer
• Part of the Tuning Pack
• FORCE_MATCH=TRUE
• Don't change the application code
• Persistent, transportable (MOS Note:751068.1), can be tested without risk
• Example:
39. PhotobyTjSedisaonUnsplash
SQL Tuning Advisor: Example
1. Problem to be solved
2. Select candidate statement to tune
3. Get SQL statement details
4. Create and run SQL Tuning task
5. View report results
47. SQL Profile - evaluation
• SQL Profiles should be evaluated before making them available to every user:
• Now evaluate the statement's profile in a limited user context
• If verification went fine, make it accessible to everybody
exec
:p_name:=dbms_sqltune.accept_sql_profile
task_name=>‘TASK_21944',name=>'XT_PROFILE',
category=>'TEST_ENV', FORCE_MATCH=>TRUE)
alter session set SQLTUNE_CATEGORY='TEST_ENV';
exec
dbms_sqltune.alter_sql_profile
(name=>'XT_PROFILE',
attribute_name=>'CATEGORY',value=>'DEFAULT')
50. SQL Tuning Example
6. Act on findings
• Follow 5 statistics recommendations to gather stats on 5 tables
• Result: 20x improvement!
51. Testing Tools | Database Replay
Collect execution
plans before upgrade
Compare AWR
Snapshots
Verify them with
SQL Performance
Analyzer
Regressed plans?
SQL Plan
Management
SQL Tuning Advisor
Verify functionality
and performance with
Database Replay
52. Testing Tools | Database Replay
Database Replay
Capture
Workload
Replay Workload
Compare – then restore
and run again with changes
11.2.0.4 19.7.0
Migrate
Preprocess
Workload
56. Behavior Change 19c | Automatic SPM
• SQL Plan Management: "Automatic" and enabled on Exadata
57. Behavior Change 19c | Automatic SPM
• SQL Plan Management: "Automatic" and enabled on Exadata
• If you'd like to revert to 12.1 or 12.2/18c behavior:
• https://mikedietrichde.com/2019/06/03/automatic-sql-plan-management-in-oracle-database-19c/
58. Statistics Advisor
Since Oracle 12.2.0.1, the Statistics Advisor is active
• Some customers reported high growth of entries
• Active on Standard Edition (SE2) as well
• https://mikedietrichde.com/2017/08/22/oracle-optimizer-statistics-advisor-in-oracle-database-12-2-0-1/
• If you desire to turn it off:
• BEGIN
dbms_advisor.set_task_parameter ('AUTO_STATS_ADVISOR_TASK','_AUTO_MMON_INTERVAL',9999999999);
dbms_advisor.set_task_parameter ('AUTO_STATS_ADVISOR_TASK','_AUTO_STATS_INTERVAL',9999999999);
END;
/
59. Automatic SQL Tuning Set
In Oracle 19.7.0, a Automatic SQL Tuning Set gets populated
• Some customers reported high growth and consumption in SYSAUX
• See: https://mikedietrichde.com/2020/05/28/do-you-love-unexpected-surprises-sys_auto_sts-in-oracle-19-7-0/
• If you want to disable it:
• BEGIN
DBMS_AUTO_TASK_ADMIN.DISABLE(client_name=>'Auto STS Capture Task', operation=>NULL, window_name=>NULL);
END;
/
• Task is not enabled by default from Oracle 19.8.0 on
60. Oracle 19c | _cursor_obsolete_threshold
• MOS Note: 2431353.1
High Version Counts for SQL statements (>1024) post upgrade To 12.2 and above causing
database slow performance
• Since Oracle 12.2.0.1, the old default of 1024 has been raised to 8192
• This may lead to various drastic performance issues
• Recommendation:
• Set it to the old default to avoid mutex concurrency and other issues:
_cursor_obsolete_threshold=1024
61. Oracle 19c | deferred_segment_creation
• MOS Note: 1216282.1
All sorts of issues and bugs - from corruption to mutex contention to customer scripts not
operating correctly anymore
• Recommendation:
Set it to
deferred_segment_creation=false
unless you have a valid reason
62. Oracle 19c | Adaptive Features
• Recommendation: Set optimizer_adaptive_statistics explicitly in your SPFILE
• See:
• https://mikedietrichde.com/2018/01/18/additional-info-adaptive-features-fixes-oracle-12-1-0-2/
• MOS Note: 2187449.1 - Recommendations for Adaptive Features in Oracle Database 12.1
Oracle 12.1 optimizer_adaptive_features = TRUE
Oracle 12.2
Adaptive
Statistics
optimizer_adaptive_statistics
Default: FALSE
Adaptive
Plans
optimizer_adaptive_plans
Default: TRUE
63. Oracle 19c | _sql_plan_directive_mgmt_control
• MOS Note: 2209560.1 - How To Disable SQL Plan Directive (SPD)
• In order to fully disable SQL Plan Directives, you need to set:
• _sql_plan_directive_mgmt_control=0
• Otherwise the database collects SPDs in the background, but won't use it
• Having optimizer_adaptive_statistics=false - which is the default - disables only the usage
of SPDs but not their creation
64. "Hidden" Features | Extended VARCHAR2
• Feature Introduced in Oracle 12.1.0.2
• Allow 32kB support for (N)VARCHAR2
• Allows length >4000 byte
ALTER SYSTEM set
MAX_STRING_SIZE=EXTENDED
scope=SPFILE;
CREATE TABLE Applicants
(id NUMBER GENERATED AS IDENTITY,
first_name varchar2(30),
last_name varchar2(30),
application date,
CV varchar2(32767)
);
SHUTDOWN IMMEDIATE
STARTUP UPGRADE
@?/rdbms/admin/utl32k.sql
65. "Hidden" Features | Extended VARCHAR2
• Fine print:
• Oracle Forms and Reports don't support Extended VARCHAR2
• MOS Note: 2221740.1
ORA-24920 In Oracle Forms 12.2.x when using MAX_STRING_SIZE=EXTENDED
• Above 3964 byte, data will be stored in an inline SecureFile LOB
• See: http://www.ludovicocaldara.net/dba/extended-data-types-storage/
• Be aware of performance impact
• See: https://blog.dbi-services.com/12c-extended-datatypes-better-than-clob/
66. Cool Features | Incremental Statistics
• Introduced in Oracle 11g
• DWH, Partitioning
• Synopses on disk can consume hundreds of GB
• Since Oracle Database 12.2.0.1:
New algorithm for synopses can drastically reduce space consumption
67. Cool Features | Incremental Statistics
Sales Table
May 22nd 2019
May 23rd 2019
May 18th 2019
May 19th 2019
May 20th 2019
May 21st 2019
May 24rd 2019
Global
Statistic
Global stats generated by aggregating the
original partition synopsis with the new one
A new partition is added
to the table & data is loaded
Global stats generated by aggregating partition
level statistics and synopsis
Retrieve synopsis for each of the other partitions
from SYSAUX
Gather partition statistics for new partition
Partition level stats are gathered & synopsis created
S1
S2
S3
S4
S5
S6
S7
SYSAUX Tablespace
68. Cool Features | Inc Stats Enhancement
• Enable that "changed" partitions won't be eligible for new stats generation
• Adjust the stale percentage if needed
SQL> exec
DBMS_STATS.SET_DATABASE_PREFS('INCREMENTAL_STALENESS','USE_STALE_PERCENT');
SQL> exec DBMS_STATS.SET_DATABASE_PREFS('STALE_PERCENT','12');
NEW IN
12.1
69. Cool Features | Inc Stats Enhancement
• Significantly smaller synopses with new algorithm
• Preference name: APPROXIMATE_NDV_ALGORITHM
• 'REPEAT OR HYPERLOGLOG' [Default]
• If synopses of old format exists, continue with the "old" algorithm
• If no or new synopses exist, use the "new" algorithm
• 'ADAPTIVE SAMPLING'
• Force to use pre-12.2 adaptive sampling algorithm – delete existing 12.2 synopses and regather - "old behavior"
• 'HYPERLOGLOG'
• Force to use the new algorithm
• Lab example:
• 8TB table with 84 partitions
• → Size in Oracle 12.1.0.2: 167 GB
• → Size in Oracle 12.2.0.1: 7 GB
NEW IN
12.2
70. Cool Features | Inc Stats Enhancement
• Preference: HYPERLOGLOG
• Replace "old" with "new"
- Recommended approach
• Preference: REPEAT OR HYPERLOGLOG
• Coexistence of "old" and "new"
- Issues seen at customers with stats gathering
SALESSYSAUX Tablespace
P2
P1 has a synopsis
in pre-12.2 format
New synopsis for P1 in
compact 12.2 format
Partitioned
Table
P1
exec dbms_stats.set_table_prefs(owner, table,
'approximate_ndv_algorithm', 'hyperloglog');
exec dbms_stats.set_table_prefs(owner, table,
'incremental_staleness', 'NULL')
SALESSYSAUX Tablespace
P2
P1 has a synopsis
in pre-12.2 format
Partitioned
Table
P1
exec dbms_stats.set_table_prefs(owner, table,
'approximate_ndv_algorithm', 'hyperloglog');
exec dbms_stats.set_table_prefs(owner, table,
'incremental_staleness', 'allow_mixed_format')
Synopsis forP2 in
compact12.2 format
NEW IN
12.2