Percona Live - Dublin 01 my sql ha-mysql-clusters

Copyright © 2016, Oracle and/or its aﬃliates. All rights reserved. |
MySQL 5.7 / 8 Update
High Availability
InnoDB Cluster and NDB Cluster
Mark Swarbrick

Copyright © 2016, Oracle and/or its aﬃliates. All rights reserved.

Safe Harbor Statement
The following is intended to outline our general product direcSon. It is intended for
informaSon purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or funcSonality, and should not be relied upon
in making purchasing decisions. The development, release, and Sming of any features or
funcSonality described for Oracle’s products remains at the sole discreSon of Oracle.
2

A Modern Database for the Digital Age
Digital Disruptors & Large Enterprises Rely on MySQL to Innovate
3

They Scale
with MySQL
Mobile Network
Suppor=ng Over 800
Million Subscribers
1.7 Billion Ac=ve Users
100 TB of User Data for PayPal
IDs Processed for 1
Billion Ci=zens
850 Million Candy
Crush Game Plays/Day
2 Billion Events/Day for
Booking.com
4

Euronext Markets TransacSon Clearing
•  Universal Clearing System: compensaSon engine for the Euronext cash and
derivaSves markets in the Euro zone
•  Highly mission criScal ﬁnancial applicaSon
•  ImplementaSon of an Open Source stack with MySQL Enterprise EdiSon
•  1 to 2 million daily business transacSons – tested for 6+ million
•  Geographical redundancy
•  DrasSc TCO reducSon & architecture simpliﬁcaSon
•  Increased performance, scalability & reliability
5
Implemented by Atos

MySQL 5.7: Designed for Web, Mobile & Cloud ApplicaSons
6
Hybrid Database
Best of both worlds: RDBMS benefits
+ flexible management of documents
Save Time, Efforts & Costs
Numerous new features and
enhancements improve manageability
Scalable Founda=on for Your Business
MySQL 5.7 is 3x faster than MySQL 5.6. Provides a reliable
& scalable foundaSon for your applicaSons
Enhanced Security
Secure by default. Advanced security features
for increased protecSon & compliance
MySQL
5.7

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 7
Hybrid Database: Rock Solid Reliability + Flexibility
MySQL 5.7

JSON Support
MySQL
Document
Store
RDBMS
Proven, transac=onal, secure
Complex JOINs and queries
Extensive opera=onal tools
NoSQL Solu=ons
Flexible. Easy-to-use.
Schema-less document storage
Modern Applica=ons
Agile DevOps with robust
data protec=on & security
Hybrid Database
No trade-offs, best of both
worlds. ACID properSes &
reliability of RDMS + flexible
document management

MySQL Cluster 7.5
8

MySQL Cluster 7.5
• Node Sizes of 128TB
• Read OpSmized Tables
• Global Tables
Capacity and
Scale Out
• MySQL 5.7
• JSON Data Type
• Generated Columns
• Records-Per-Key
OpSmizaSon
Improved
SQL
• Improved ReporSng
• Improved Logging
• Improved Visibility
• Improved Debugging
• Improved Restore
Management
RC

MySQL 8.0
10

MySQL 8 – Improvements across the board
•  ReplicaSon
•  InnoDB
•  OpSmizer
•  Security
•  Performance Schema
•  GIS
•  Triggers
•  ParSSoning
•  New! SYS Schema
•  New! JSON
•  Performance
11
200+ new
features
In total!

The character set for the Web

MySQL 8.0: UTF-8
•  New! Support for the latest Unicode 9.0
•  Per-country collaSon rules
– Accent SensiSve
– Case SensiSve
•  Based on UCA DUCET
•  UTF8MB4 as the default character set
– Project started in MySQL 5.7
– Many improvements to reduce performance impact

MySQL 8.0: Invisible Indexes
•  Indexes are “hidden” to the MySQL OpSmizer
– Not the same as “disabled indexes”
– Contents are fully up to date and maintained by DML
•  Two use cases:
– Soq Delete (Recycle Bin)
– Staged Rollout

Feature Request
from DBAs

MySQL 8.0: Security Roles
•  Fully FuncSon, Flexible, Properly Architected Roles
•  Create and Drop Roles, Grant to Roles
•  Grant Roles to Roles, Grant Roles to Users
•  Limit Hosts that can use roles, Deﬁne Default Roles
•  Decide what roles are applicable during a session
•  And even visualize Roles with SQL funcSon ROLES_GRAPHML()

MySQL 8.0: Atomicity in Privileges
•  Privilege Tables now 100% InnoDB
•  User Management DDLs Atomic
– CREATE USER
– ALTER USER
– RENAME USER
– DROP USER
– GRANT
– REVOKE

•  Increased Reliability
•  Built on InnoDB
– No FRM files
– No DB.OPT files
– No TRG files
– No TRN files
– No PAR files
– MyISAM not required
•  Atomic DDL
– Bever ReplicaSon
– Simplifies server edge cases
•  MDL for Foreign Keys
•  Flexible Metadata API
– Easier path to adding new features
•  Bever cross-plaworm experience
– No dependencies on filesystem
semanScs
16
MySQL 8.0: NaSve Data DicSonary

Almost all organizaSons require their most
criScal systems to be highly available
17
100%

InnoDB Cluster
18
or
NDB Cluster

•  MySQL InnoDB Cluster
– Easy HA built into MySQL 5.7+
– Write consistency
– Read Scalability
– App failover using MySQL Router
– Management via mysqlsh
– InnoDB storage engine

•  MySQL NDB Cluster
– In memory database
– AutomaSc sharding
– NaSve access via several API
– Read/write consistency
– Read/write scalability
– NDB storage engine
How Do The Two Compare I

How Do The Two Compare II
MySQL InnoDB Cluster MySQL NDB Cluster
Storage Engine InnoDB NDBCLUSTER
Distributed Architecture Shared nothing Shared nothing
Consistency Model Fairly Strong Consistency Strong Consistency
Sharding No Yes
ArbitraSon No Yes
Load Balancing No Yes
NoSQL APIs MySQL Document Store NaSve NDB API
OperaSonal Complexity Medium High
AdministraSon Standard (MySQL) Custom (MySQL + NDB)

InnoDB Cluster
21

Introducing …
22

Scale-Out
Ease-of-Use
Out-of-Box
SoluSon
MySQL

A single product: MySQL
•  All components created together
•  Tested together
•  Packaged together
Flexible and Modern
•  C++ 11
•  Protocol Buﬀers
•  Developer friendly
MySQL InnoDB Cluster – Goals
Easy to use
•  A single client: MySQL Shell
•  Easy packaging
•  Homogenous servers
Scale-out
•  Sharded clusters
•  Federated system of N replica sets
•  Each replica set manages a shard

4
Rollout will happen in four steps.
25

Read Scale-Out
Async ReplicaSon + Auto Failover
Write Scale-Out
Sharding
S1
S2
S3
S4
MySQL Vision – 4 Steps
Timeline
MySQL Document Store
RelaSonal & Document Model
MySQL HA
Out-Of-Box HA

InnoDB Cluster
27
App Servers with
MySQL Router
MySQL Group ReplicaSon
MySQL Shell
Setup, Manage,
Orchestrate
“High Availability becomes a core
ﬁrst class feature of MySQL!”

MySQL InnoDB Cluster: Vision

“A single product — MySQL — with high availability and scaling
features baked in; providing an integrated end-to-end soluIon
that is easy to use.”

•  One Product: MySQL
– All components created together
– Tested together
– Packaged together
•  Easy to Use
– One client: MySQL Shell
– Easy packaging
– Integrated orchestraSon
– Homogenous servers
•  Flexible and Modern
– SQL and NoSQL together
– Protocol Buﬀers
– Developer friendly
•  Support Read/Write Scale Out
– Sharded clusters
– Federated system of N replica sets
– Each replica set manages a shard
29
MySQL InnoDB Cluster: Goals

MySQL Connector
ApplicaSon
MySQL Router
MySQL Connector
ApplicaSon
MySQL Router
MySQL Shell
MySQL Connector
ApplicaSon
MySQL Router
MySQL Connector
ApplicaSon
MySQL Router
MySQL InnoDB Cluster: High Level Architecture
MySQL
InnoDB
cluster
MySQL Enterprise Monitor
…


MySQL
Group ReplicaSon
Na=vely distributed and highly available replica sets

MySQL Group ReplicaSon: What Is It?
•  Group ReplicaSon library
– ImplementaSon of Replicated Database State Machine
theory
•  MySQL GCS is based on Paxos (variant of Mencius)
– Provides virtually synchronous replicaSon for MySQL 5.7+
– Supported on all MySQL plaKorms
•  Linux, Windows, Solaris, OSX, FreeBSD

“MulI-master update anywhere replicaIon plugin for MySQL with
built-in conﬂict detec1on and resolu1on, automa1c distributed
recovery, and group membership.”

32
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate

MySQL Group ReplicaSon: What Does It Provide?
•  A highly available distributed MySQL database service
– Removes the need for manually handling server fail-over
– Provides distributed fault tolerance
– Enables AcSve/AcSve update anywhere setups
– Automates reconﬁguraSon (adding/removing nodes, crashes, failures)
– AutomaScally detects and handles conﬂicts

33

MySQL Group ReplicaSon: Architecture
Node Types
R: Traﬃc routers/proxies: mysqlrouter, haproxy, sqlproxy, ...
M: mysqld nodes parScipaSng in Group ReplicaSon
34

MySQL Server: Full Stack


MySQL Router
Transparent applica=on connec=on rou=ng

MySQL Router: 2.1
•  NaSve support for InnoDB clusters
– Understands Group ReplicaSon topology
– USlizes metadata schema stored on each member
•  Bootstraps itself and sets up client rouSng for the InnoDB cluster
•  Allows for intelligent client rouSng into the InnoDB cluster
•  Supports mulS-master and single primary modes
•  Core improvements
– Logging
– Monitoring
– Performance
– Security
37
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
labs.mysql.com


MySQL Shell
Single tool for development, setup,
management, orchestra=on, and monitoring

MySQL Shell
39
A single uniﬁed client for all administra=ve and opera=ons tasks
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
•  MulS-Language: JavaScript, Python, and SQL
– Naturally scriptable
•  Supports both Document and RelaSonal models
•  Exposes full Development and Admin API

”MySQL Shell provides the developer and DBA with a single intuiIve,
ﬂexible, and powerfull interface for all MySQL related tasks!”

MySQL Shell: Admin API
•  mysql-js> dba.help()
•  The global variable 'dba' is used to access the
MySQL AdminAPI
•  Perform DBA operaSons
– Manage MySQL InnoDB clusters
•  Create clusters
•  Deploy MySQL instances
•  Get cluster info
•  Start/Stop MySQL Instances
•  Validate MySQL instances …
Database Administra=on Interface
labs.mysql.com
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate

MySQL Shell – Deploy MySQL Instances

shell> mysqlsh

mysql-js> dba.deployLocalInstance(3306)

mysql-js> dba.deployInstance(‘hanode2:3306’)

mysql-js> dba.deployInstance(‘hanode3:3306’)

41

MySQL Shell – Create an InnoDB Cluster

mysql-js> connect root@hanode1:3306

mysql-js> cluster = dba.createCluster(‘NewAppCluster')

mysql-js> cluster.addInstance('root@hanode2:3306')

mysql-js> cluster.addInstance('root@hanode3:3306')

42

MySQL Shell – Add a MySQL Router

shell> mysqlrouter --bootstrap hanode1:3306

shell> mysqlrouter &

shell> mysqlsh --uri root@localhost:6446

43

MySQL Enterprise Monitor
•  NaSve holisSc support for InnoDB clusters
– Topology views
– Detailed metrics and graphs
– Best PracSce advice
•  Monitoring of MySQL Routers
•  Monitoring of Group ReplicaSon
44

NDB Cluster
45

Driving Database Requirements
9th April 2015 Copyright 2015, oracle and/or its aﬃliates. All rights reserved 46
Rock Solid
Availability
In-Memory
Real-Time
Performance
Extreme
Read & Write
Scalability
ElasScity

HLR / HSS
Billing, AuC, VLR
AuC, Call
RouSng, Billing
LocaSon
Updates
Pre & Post Paid
•  Massive volumes of write traﬃc
•  < 3ms database response
•  DownSme & lost transacSons = lost $
•  Extremly cost sensiSve market
MySQL Cluster in Action: http://bit.ly/oRI5tF
No Trade-Oﬀs: Cellular Network

MySQL Cluster Overview
• Memory opSmized tables with durability
• Predictable Low-Latency, Bounded Access Time REAL-TIME
• Auto-Sharding, AcSve-AcSve
• ACID Compliant, OLTP + Real-Time AnalyScs HIGH SCALE, READS + WRITES
FULLY ELASTIC
•  AcSve-AcSve, Shared nothing, no Single Point of Failure
•  Self Healing + On-Line OperaSons
99.999% AVAILABILITY
•  Key/Value + Complex, RelaSonal Queries
•  SQL + Memcached + JavaScript + Java + HTTP/REST & C++
SQL + NoSQL
•  Open Source + Commercial EdiSons, Easy to use and deploy
•  Commodity hardware + Management, Monitoring Tools
LOW TCO
•  Add and remove storage and performance capacity in seconds
•  Fully cloud enabled

User Session
Session
management
Session
management
LocaSon Updates,
Character movements, ...
•  Massive volumes of write traﬃc
•  < 3ms database response
•  DownSme & lost transacSons = lost $
•  Extremly sensiSve crowd
No Trade-Oﬀs: Massive Parallel Online Games

MySQL Cluster Architecture

MySQL Cluster Data Nodes
Clients
ApplicaSon Layer
Data Layer

Data ParSSoning / Auto-sharding
10/10/17 Copyright 2015, oracle and/or its aﬃliates. All rights reserved 51
User-id (PK) Service (PK) Data
1773467253 chat xxx
6257346892 chat xxx
1773467253 photos xxx
8235602099 reminders xxx
8437829249 loca=on xxx

1773467253 chat xxx
6257346892 chat xxx
Shard Key

Node Group
Data Node 1
•  Data transparently sharded between Node Groups
•  Each fragment acSve in one Data Node with synchronous replicaSon to
2nd Data Node in same Node Group
Data Node 2
Node Group
Data Node 3 Data Node 4

•  DBA chooses which part of
Primary Key to use as shard
key
•  Fragment for each row
decided by hashing the
sharding key
1773467253 chat xxx
6257346892 chat xxx
Shard Key
Node Group
Node Group

•  Data automaScally rearranged to use new capacity
•  Designed to be a slow background process not impacSng real-Sme
performance.
Node Group
Node Group
Node Group
On-line Scaling and ElasScity
Re-parSoning

MySQL Cluster 7.2: 1.2 Billion UPDATEs per Minute
•  NoSQL C++ API,
ﬂexaSynch benchmark
•  30 x Intel E5-2600 Intel
Servers, 2 socket, 64GB
•  ACID TransacSons, with
Synchronous ReplicaSon

hvp://www.mysql.com/why-mysql/
white-papers/mysql-cluster-
benchmarks-1-billion-writes-per-minute/ 0
5
10
15
20
25
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
MillionsofUPDATEsperSecond

MySQL NDB Cluster 7.5
• Read OpSmized
Tables
• Fully Replicated
Capacity and
Scale Out
• MySQL 5.7
• JSON Data Type
• Generated Columns
• Records-Per-Key
OpSmizaSon
Improved
SQL
• Improved ReporSng
• Improved Logging
• Improved Visibility
• Improved Restore
Management
RC

•  Reading from backup allows to
read from any copy
•  Previously all reads were directed
towards the primary fragment only
ConﬁdenSal – Oracle Internal/Restricted/Highly Restricted 59
MySQL NDB 7.5: Reading from backup
Node Group

Next Steps
Learn More
•  www.mysql.com/cluster
•  hvps://dev.mysql.com/doc/mysql-innodb-cluster/en/
•  AuthenSc MySQL Curriculum: hvp://oracle.com/educaSon/mysql
Try it Out
•  dev.mysql.com/downloads/cluster/
7.4 GA and 7.5 DMR 1
•  hvps://labs.mysql.com/
Let us know what you think
•  hvp://mysqlhighavailability.com/
•  forums.mysql.com/list.php?25

Thank you!

Percona Live - Dublin 01 my sql ha-mysql-clusters

More Related Content

Percona Live - Dublin 01 my sql ha-mysql-clusters