MySQL performance webinar

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Performance Tuning
Abel Flórez
MySQL Technical Account Manager
September 2015

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
3

Program Agenda
General Info
InnoDB
Query Cache – Older systems
Threads and Table Cache
Session Level
Tools for monitoring, analyzing, and tuning queries
1
2
3
4
4
5
6

Program Agenda
General Info
InnoDB
Session Level
1
2
3
5
4
5
6

Current Settings
• my.cnf / my.ini
7
# The following options will be passed to MySQL clients
[client]
#password = your_password
#port = 5622
#socket = /tmp/mysql.5.6.22.sock
# The MySQL server
[mysqld]
####################
# specific to this test instance
port = 5622
socket = /tmp/mysql.5.6.22.sock
log-error = /usr/local/mysql/data/error.log
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
safe-user-create
####################
server-id = 1
sql_mode = NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
table_open_cache = 4000

Current Status
mysql> SHOW GLOBAL STATUS;
8
mysql> SHOW GLOBAL STATUS;
+--------------------------------------+---------------+
| Variable_name | Value |
+--------------------------------------+---------------+
| Aborted_clients | 9363 |
| Aborted_connects | 15925 |
| Binlog_cache_disk_use | 15684 |
| Binlog_cache_use | 272758675 |
| Bytes_received | 1219816389366 |
| Bytes_sent | 5227022143540 |
| Com_admin_commands | 12803263 |
| Com_assign_to_keycache | 0 |
| Com_alter_db | 0 |
| Com_alter_db_upgrade | 0 |
| Com_alter_event | 0 |
| Com_alter_function | 0 |
| Com_alter_procedure | 0 |
| Com_alter_server | 0 |
| Com_alter_table | 37 |
| Com_alter_tablespace | 0 |
| Com_analyze | 0 |
| Com_begin | 33025640 |
| Com_binlog | 0 |
| Com_call_procedure | 0 |
| Com_change_db | 1 |
| Com_change_master | 0 |
| Com_check | 0 |
| Com_checksum | 0 |
| Com_commit | 33025622 |

COM_*
• Counters for each command
9
| Com_resignal | 0 |
| Com_revoke | 0 |
| Com_revoke_all | 0 |
| Com_rollback | 0 |
| Com_rollback_to_savepoint | 0 |
| Com_savepoint | 0 |
| Com_select | 1530076294 |
| Com_set_option | 18004985 |
| Com_signal | 0 |
| Com_show_authors | 0 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 27498 |
| Com_show_charsets | 1 |
| Com_show_collations | 39 |
| Com_show_contributors | 0 |
| Com_show_create_db | 3 |
| Com_show_create_event | 0 |
| Com_show_create_func | 1120 |
| Com_show_create_proc | 14499 |
| Com_show_create_table | 183538 |
| Com_show_create_trigger | 0 |
| Com_show_databases | 435 |
| Com_show_engine_logs | 0 |
| Com_show_engine_mutex | 0 |
| Com_show_engine_status | 1180106 |
| Com_show_events | 0 |
| Com_show_errors | 0 |
| Com_show_fields | 2915144 |
| Com_show_function_status | 29 |
https://dev.mysql.com/doc/refman/5.6/en/
server-status-variables.html
show global status like 'Com_%';

Additional Commands
• Connections and Max_used_connections
10
https://dev.mysql.com/doc/refman/5.6/en/server-status-variables.html

Additional Commands
• Queries and Questions
11

Additional Commands
• Slow_queries
12

Additional Commands
• Slow_queries
• Handler_read_*
13
Handler_read_first
Handler_read_key
Handler_read_next
Handler_read_prev
Handler_read_rnd
Handler_read_rnd_next

Program Agenda
General Info
InnoDB
Session Level
1
2
3
14
4
5
6

How is InnoDB doing?
• Efficiency
– InnoDB_buffer_pool_reads_requests
– InnoDB_buffer_pool_reads
15
Formula:
1 - (Innodb_buffer_pool_read / Innodb_buffer_pool_read_requests)
= buffer pool hit ratio

innodb_buffer_pool_size
• Global
• Caches data and indexes
• Larger values reduce IO
• 80% max
16
https://dev.mysql.com/doc/refman/5.6/en/innodb-buffer-pool.html

innodb_log_file_size
• Size of file on disk
• Larger the file, the less checkpoint activity
• Size for each log file
• Max combined log file size
– 4G <= 5.5
– 512G = 5.6
• Larger the log file ➯ longer recovery time
17
http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html

innodb_log_buffer_size
• Buffer for writing to log files on disk
• Large transactions? Write heavy?
– Yes – increasing can help with IO
• Default: 8M
18

innodb_file_per_table
• .ibd files
• Default: Off <= 5.6.5 > On
• TRUNCATE for a table is faster
• Can monitor table size on the file system
• Can store specific tables on different storage devices
• (5.6) Transportable tablespaces
19

innodb_flush_log_at_trx_commit
• Performance vs. Durability
• 1 – Default
– ACID Compliant
– InnoDB log buffer written to log file and flush to disk at each commit.
• 2
– Can lose up to 1 sec of transactions
– InnoDB log buffer written to log file at each commit, flush to disk once per second.
• 0
– Can lose up to 1 sec of transactions
– InnoDB log buffer written to log file and flush to disk once per sec.
20
>= 5.6.6
innodb_flush_log_at_timeout

innodb_buffer_pool_instances
• 5.6
• Problem: contention for the buffer pool resources
• For systems with large multi-GB buffer pools
• Total buffer pools size divided between this many instances
• Each instance
– Manages its own resources
– Has its own mutex
21

innodb_stats_persistent
• Specifies whether InnoDB index statistics are persisted to disk.
22

Program Agenda
General Info
InnoDB
Query Cache
Session Level
1
2
3
23
4
5
6

How is the QC doing?
• Qcache_total_blocks
• Qcache_free_blocks
• Qcache_lowmem_prunes
24
https://dev.mysql.com/doc/refman/5.6/en/query-cache-status-and-maintenance.html
SHOW VARIABLES LIKE 'have_query_cache';

How is the QC doing? (cont.)
• Qcache_hits
• Qcache_inserts
• Qcache_not_cached
• Qcache_queries_in_cache
25
https://dev.mysql.com/doc/refman/5.6/en/query-cache-status-and-maintenance.html

• Query Cache Hit Rate
– Higher the better
• Invalidating queries
– Bigger the diff – the better
Formula:
Qcache_hits / (Qcache_hits + Com_select) = Query Cache Hit Rate
Formula:
Qcache_inserts << Com_select
QC Data Into Info
26

query_cache_size
• Global
• Allocated all at once
• Default: disabled
• Default size:
– 0 < 5.6.8
– 1M >= 5.6.8
27
https://dev.mysql.com/doc/refman/5.6/en/query-cache-configuration.html
https://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html

query_cache_type
• Global – but can be session
• 3 options
– 0 “Off”
– 1 “On”
– 2 “On Demand”
• Default:
– 1 < 5.6.8
– 0 >= 5.6.8
28

Program Agenda
General Info
InnoDB
Session Level
1
2
3
29
4
5
6

Threads
• Threads == Connections
• Thread Cache
– Threads_cached
– Threads_connected
– Threads_created
30
Formula:
Threads_created / Connections = Thread cache miss rate
https://dev.mysql.com/doc/refman/5.6/en/connection-threads.html

thread_cache_size
• Global but grows as needed
• Number to be cached until needed
• May increase performance with lots of new connections
– Or maybe not
31
Formula:
100 – ((Threads_created / Connections) * 100) =
Thread cache efficiency
https://dev.mysql.com/doc/refman/5.6/en/connection-threads.html

table_cache
• table_cache <= 5.1.3 > table_open_cache
• Expands as needed
• Number of open tables for all connections
• Watch Opened_tables
32
https://dev.mysql.com/doc/refman/5.6/en/table-cache.html

Other “table cache” Options
• table_definition_cache (intro 5.1.3)
– Number of table definitions (.frm file) to hold in the cache
– InnoDB
• Soft limit for open table instances in data dictionary
• Soft limit for number of InnoDB file per table tablespaces open at 1 time
• table_open_cache_instances (intro 5.6)
– Reduces contention between sessions
– Default 1
• Consider 8 or 16 on 16+ core system
33

Program Agenda
General Info
InnoDB
Session Level
1
2
3
34
4
5
6

Temporary Tables
35
max_heap_table_size
• Dynamic
• Set maximum size of all
MEMORY tables
tmp_table_size
• Dynamic
• In-memory – MEMORY table
• Max size of internal in-
memory temp tables
• Becomes on-disk MyISAM
table

sort_buffer_size
• Dynamic
• Doing a sort – allocates a buffer this size
• Helps with ORDER BY and/or GROUP BY
• Watch Sort_merge_passes
36

Program Agenda with Highlight
General Info
InnoDB
Session Level
1
2
3
37
4
5
6

Tools
Performance schema
MySQL sys schema
EXPLAIN & Optimizer trace
MySQL Enterprise Monitor (MEM)

Performance Schema
• events_statements_history
events_statements_history_long
– Most recent statements executed
• events_statements_summary_by_digest
– Summary for similar statements (same statement digest)
• file_summary_by_event_name
– Interesting events: wait/io/file/innodb/innodb_data_file
• table_io_waits_summary_by_table
table_io_waits_summary_by_index_usage
– Statistics on storage engine access per table and index
Some useful tables
http://dev.mysql.com/doc/refman/5.6/en/performance-schema.html

Performance Schema
• Normalization of queries to group statements that are similar to be
grouped and summarized:
SELECT * FROM orders WHERE o_custkey=10 AND o_totalprice>20
SELECT * FROM orders WHERE o_custkey = 20 AND o_totalprice > 100
SELECT * FROM orders WHERE o_custkey = ? AND o_totalprice > ?
• events_statements_summary_by_digest
DIGEST, DIGEST_TEXT, COUNT_STAR, SUM_TIMER_WAIT, MIN_TIMER_WAIT, AVG_TIMER_WAIT,
MAX_TIMER_WAIT, SUM_LOCK_TIME, SUM_ERRORS, SUM_WARNINGS, SUM_ROWS_AFFECTED,
SUM_ROWS_SENT, SUM_ROWS_EXAMINED, SUM_CREATED_TMP_DISK_TABLES, SUM_CREATED_TMP_TABLES,
SUM_SELECT_FULL_JOIN, SUM_SELECT_FULL_RANGE_JOIN, SUM_SELECT_RANGE, SUM_SELECT_RANGE_CHECK,
SUM_SELECT_SCAN, SUM_SORT_MERGE_PASSES, SUM_SORT_RANGE, SUM_SORT_ROWS, SUM_SORT_SCAN,
SUM_NO_INDEX_USED, SUM_NO_GOOD_INDEX_USED, FIRST_SEEN, LAST_SEEN
Statement digest

Performance Schema
• Tables:
events_statements_current (Current statement for each thread)
events_statements_history (10 most recent statements per thread)
events_statements_history_long (10000 most recent statements)
• Columns:
THREAD_ID, EVENT_ID, END_EVENT_ID, EVENT_NAME, SOURCE, TIMER_START, TIMER_END, TIMER_WAIT,
LOCK_TIME, SQL_TEXT, DIGEST, DIGEST_TEXT, CURRENT_SCHEMA, OBJECT_TYPE, OBJECT_SCHEMA,
OBJECT_NAME, OBJECT_INSTANCE_BEGIN, MYSQL_ERRNO, RETURNED_SQLSTATE, MESSAGE_TEXT, ERRORS,
WARNINGS, ROWS_AFFECTED, ROWS_SENT, ROWS_EXAMINED, CREATED_TMP_DISK_TABLES,
CREATED_TMP_TABLES, SELECT_FULL_JOIN, SELECT_FULL_RANGE_JOIN, SELECT_RANGE, SELECT_RANGE_CHECK,
SELECT_SCAN, SORT_MERGE_PASSES, SORT_RANGE, SORT_ROWS, SORT_SCAN, NO_INDEX_USED,
NO_GOOD_INDEX_USED, NESTING_EVENT_ID, NESTING_EVENT_TYPE
Statement events

MySQL sys Schema
• Started as a collection of views, procedures and functions, designed to
make reading raw Performance Schema data easier
• Implements many common DBA and Developer use cases
• Now bundled within MySQL Workbench 6.1
• Available on GitHub
– https://github.com/MarkLeith/mysql-sys
• Examples of very useful functions:
– format_time() , format_bytes(), format_statement()

MySQL sys Schema
statement_analysis: Lists a normalized statement view with aggregated
statistics, mimics the MySQL Enterprise Monitor Query Analysis view,
ordered by the total execution time per normalized statement
mysql> select * from statement_analysis limit 1G
*************************** 1. row ***************************
query: INSERT INTO `mem__quan` . `nor ... nDuration` = IF ( VALUES ( ...
db: mem
full_scan:
exec_count: 1110067
err_count: 0
warn_count: 0
total_latency: 1.93h
max_latency: 5.03 s
avg_latency: 6.27 ms
Example
lock_latency: 00:18:29.18
rows_sent: 0
rows_sent_avg: 0
rows_examined: 0
rows_examined_avg: 0
tmp_tables: 0
tmp_disk_tables: 0
rows_sorted: 0
sort_merge_passes: 0
digest: d48316a218e95b1b8b72db5e6b177788!
first_seen: 2014-05-20 10:42:17

EXPLAIN & Optimizer trace
• EXPLAIN shows the selected plan
• TRACE shows WHY the plan was selected:
– Alternative plans
– Estimated costs
– Decisions made
• JSON format

Optimizer Trace: Example
SET optimizer_trace= "enabled=on", end_markers_in_json=on;
SELECT * FROM t1,t2 WHERE f1=1 AND f1=f2 AND f2>0;
SELECT trace INTO DUMPFILE "<filename>"
FROM information_schema.optimizer_trace;
SET optimizer_trace="enabled=off";
QUERY SELECT * FROM t1,t2 WHERE f1=1 AND f1=f2 AND f2>0;
TRACE “steps”: [ { "join_preparation": { "select#": 1,… } … } …]
MISSING_BYTES_BEYOND_MAX_MEM_SIZE 0
INSUFFICIENT_PRIVILEGES 0
http://dev.mysql.com/doc/internals/en/optimizer-tracing.html

MySQL Enterprise Monitor

Performance Issues Exposed Via Monitoring
• Unplanned configuration values
• Going to disk too often (wasted IOPs)
• Inefficient query plans
• Statements issued too often

Wrong/Poor Configuration
• MySQL version in use
• InnoDB buffer pool size
• InnoDB transaction log
and log buffer sizes
• Thread cache

Throughput

Concurrency

Query Analyzer
Proxy Connector Plugin 5.6 Performance
Schema
Supported Server
Versions
Any Any > 5.6.14
Supported Clients Any JDBC, PHP, ADO.Net Any
Complexity High Medium Low
Value/Richness of Data Low Medium High

Query Analyzer Details

Queries and databases

Missing Indexes

High Rate of Low Latency Statements

Statements With Errors and/or Warnings

Query Response Time index
https://en.wikipedia.org/wiki/Apdex

High Average Latency Statements

New, Un-optimized Statements

Extra Information from >= 5.6 and Performance Schema

High Average Examined Rows
Only Available
with >= 5.6 and
Performance Schema!

On-Disk Temporary Tables
Only Available
with >= 5.6 and
Performance Schema!

How Do I Get it?
• Part of the MySQL Enterprise Subscription
• Trial Licenses
• Oracle Software Delivery Cloud
➯ https://edelivery.oracle.com/
➯ “MySQL Database Product Pack”

Thank you for attending!
64

MySQL performance webinar

More Related Content

MySQL performance webinar