MySQL Day Paris 2016 - MySQL HA: InnoDB Cluster and NDB Cluster

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
MySQL High Availability
InnoDB Cluster and NDB Cluster
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
@freshdaz

Safe Harbor Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.
2

Almost all organizations require
their most critical systems to be
highly available
3
100%

High Availability
Protecting business critical data.
Servers can crash, hardware can fail - but
business must go on.
Existing solutions are complex, incomplete
and hard to use.
4

Scale-Out
Allow database to grow with your business.
Be prepared for major success.
Scale-out without sacrificing transactional
guarantees is hard.
5

MySQL High-Availability & Scale-Out Solutions

InnoDB Cluster
7
or
NDB Cluster

• MySQL InnoDB Cluster
– Easy HA built into MySQL 5.7+
– InnoDB storage engine
– Write consistency
– Read Scalability
– App failover using MySQL Router
– Management via MySQL shell
(mysqlsh)
• MySQL NDB Cluster
– In memory distributed database
– NDB storage engine
– Automatic sharding
– Native access via several API
– Read/write consistency
– Read/write scalability
– Management via MCM/mgm-tool
How do the 2 compare ?

How do the 2 compare ? (cont.)
MySQL InnoDB Cluster MySQL NDB Cluster
Storage Engine InnoDB NDBCLUSTER
Distributed Architecture Shared nothing Shared nothing
Consistency Model Weak Consistency Strong Consistency
Sharding No* Yes
Network failure protection Quorum Arbitration
Load Balancing No* Yes
NoSQL APIs MySQL Document Store Native NDB APIs
Operational Complexity Medium High
Administration Standard (MySQL) Custom (MySQL + NDB)

InnoDB Cluster
10

InnoDB Cluster
11
App Servers with
MySQL Router
MySQL Group Replication
MySQL Shell
Setup, Manage,
Orchestrate
“High Availability becomes a core
first class feature of MySQL!”

• One Product: MySQL
– All components created together
– Tested together
– Packaged together
• Easy to Use
– One client: MySQL Shell
– Easy packaging
– Integrated orchestration
– Homogeneous servers
• Flexible and Modern
– SQL and NoSQL together
– Protocol Buffers
– Developer friendly
• Support Read/Write Scale Out
– Sharded clusters
– Federated system of N replica sets
– Each replica set manages a shard
12
MySQL InnoDB Cluster: Goals

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 13
MySQL InnoDB Cluster: Vision
“A single product — MySQL — with high availability
and scaling features baked in; providing an
integrated end-to-end solution that is easy to use.”

Scale-Out
High Performance
Ease-of-Use
Built-in HA
Out-of-Box Solution
Everything Integrated
MySQL
InnoDB
cluster

Read Scale-Out
Async Replication + Auto Failover
Write Scale-Out
Sharding
S1
S2
S3
S4
MySQL Vision – 4 Steps
Timeline
MySQL Document Store
Relational & Document Model
MySQL HA
Out-Of-Box HA

MySQL
InnoDB
cluster
MySQL InnoDB Cluster – Architecture - S2
M
M M
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL Shell
HA
Group Replication

S1 S2 S3 S4 S…
M
M M
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL Shell
HA
MySQL InnoDB Cluster – Architecture - S3 MySQL
InnoDB
cluster
Read-Only Slaves

S1 S2 S3 S4 S…
M
M M
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL Shell
HA
ReplicaSet(Shard1)
S1 S2 S3 S4 S…
M
M M
MySQL Connector
Application
MySQL Router
HA
ReplicaSet(Shard2)
S1 S2 S3 S4 S…
M
M M
HA
ReplicaSet(Shard3)
MySQL Connector
Application
MySQL Router
MySQL InnoDB Cluster – Architecture - S4
…
MySQL
InnoDB
cluster

MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL Shell
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL InnoDB Cluster: High Level Architecture
MySQL
InnoDB
cluster
MySQL Enterprise Monitor
…

MySQL InnoDB Cluster: The End Goal

Natively distributed and highly available replica sets

MySQL Group Replication: What Is It?
• Group Replication plugin
– Implementation of Replicated Database State Machine theory
• MySQL GCS is based on Paxos (variant of Mencius)
– Provides virtually synchronous replication for MySQL 5.7+
– Supported on all MySQL platforms
• Linux, Windows, Solaris, OSX, FreeBSD
“Multi-master update anywhere replication plugin for MySQL with built-in
conflict detection and resolution, automatic distributed recovery,
and group membership.”
22
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
RC

MySQL Group Replication: What Does It Provide?
• A highly available distributed MySQL database service
– Removes the need for manually handling server fail-over
– Provides distributed fault tolerance
– Enables Active/Active update anywhere setups
– Automates reconfiguration (adding/removing nodes, crashes, failures)
– Automatically detects and handles conflicts
23

MySQL Group Replication: Architecture
Node Types
R: Traffic routers/proxies: mysqlrouter, haproxy, sqlproxy, ...
M: mysqld nodes participating in Group Replication
24

MySQL Group Replication: Performance
26

MySQL Router
Transparent application connection routing

MySQL Router: 2.1
• Native support for InnoDB clusters
– Understands Group Replication topology
– Utilizes metadata schema stored on each member
• Bootstraps itself and sets up client routing for the InnoDB cluster
• Allows for intelligent client routing into the InnoDB cluster
• Supports multi-master and single primary modes
• Core improvements
– Logging
– Monitoring
– Performance
– Security
28
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
labs.mysql.com

MySQL Shell
Single tool for development, setup,
management, orchestration, and monitoring

MySQL Shell
30
A single unified client for all administrative and operations tasks
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
• Multi-Language: JavaScript, Python, and SQL
– Naturally scriptable
• Supports both Document and Relational models
• Exposes full Development and Admin API
”MySQL Shell provides the developer and DBA with a single intuitive,
flexible, and powerfull interface for all MySQL related tasks!”

MySQL Shell: Admin API
• mysql-js> dba.help()
• The global variable 'dba' is used to access the
MySQL AdminAPI
• Perform DBA operations
– Manage MySQL InnoDB clusters
• Create clusters
• Deploy MySQL instances
• Get cluster info
• Start/Stop MySQL Instances
• Validate MySQL instances …
Database Administration Interface
labs.mysql.com
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate

MySQL Shell – Deploy MySQL Instances
  shell> mysqlsh
  mysqljs> dba.deployLocalInstance(3306)
  mysqljs> dba.deployInstance(‘hanode2:3306’)
  shell> mysqlsh
  mysqljs> dba.deployLocalInstance(3306)
32
https://www.youtube.com/watch?v=JWy7ZLXxtZ4

MySQL Shell – Create an InnoDB Cluster
  mysqljs> connect root@hanode1:3306
  mysqljs> cluster = dba.createCluster(‘NewAppCluster')
  mysqljs> cluster.addInstance('root@hanode2:3306')
  mysqljs> connect root@hanode1:3306
  mysqljs> cluster = dba.createCluster(‘NewAppCluster')
33

MySQL Shell – Add a MySQL Router
  shell> mysqlrouter bootstrap hanode1:3306
  shell> mysqlrouter &
  shell> mysqlsh uri root@localhost:6446
  shell> mysqlrouter bootstrap hanode1:3306
  shell> mysqlrouter &
  shell> mysqlsh uri root@localhost:6446
34

MySQL Enterprise Monitor
• Native holistic support for InnoDB clusters
– Topology views
– Detailed metrics and graphs
– Best Practice advice
• Monitoring of MySQL Routers
• Monitoring of Group Replication
35
https://www.mysql.com/products/enterprise/monitor.html

Sharded InnoDB Clusters
• Group Replication
– Each shard is a highly available replica set
• MySQL Router
– Manages shard mappings and related metadata
– Manages client routing
– Provides cross shard execution framework
• MySQL Shell
–Exposes management and orchestration features
36
* Download InnoDB Cluster Preview Release from labs.mysql.com

NDB Cluster
37

Driving Database Requirements
38
Rock Solid
Availability
In-Memory
Real-Time
Performance
Extreme
Read & Write
Scalability
Elasticity

HLR / HSS
Billing, AuC, VLR
AuC, Call
Routing, Billing
Location
Updates
Pre & Post Paid
• Massive volumes of write traffic
• < 3ms database response
• Downtime & lost transactions = lost $
• Extremely cost sensitive market
MySQL Cluster in Action: http://bit.ly/oRI5tF
No Trade-Offs: Cellular Network

MySQL Cluster Overview
42
FULLY ELASTICFULLY ELASTIC
• Add and remove storage and performance capacity in seconds
• Fully cloud enabled

NoSQL Access to MySQL Cluster data
Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps
JPA
Cluster JPA
PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached
MySQL JNI Node.JS mod_ndb ndb_eng
NDB API (C++)
MySQL Cluster Data Nodes

MySQL Cluster Architecture
Clients
Application Layer
Data Layer
44
Management Layer

Data Partitioning / Auto-sharding
45
User-id (PK) Service (PK) Data
1773467253 chat xxx
6257346892 chat xxx
1773467253 photos xxx
8235602099 reminders xxx
8437829249 location xxx

46
1773467253 chat xxx
6257346892 chat xxx
Shard Key

47
1773467253 chat xxx
6257346892 chat xxx
Shard Key

Node GroupNode Group
Data Node 1Data Node 1
• Data transparently sharded between Node Groups
• Each fragment active in one Data Node with synchronous replication to 2nd
Data Node in same Node Group
48
Data Node 2Data Node 2
Data Node 3Data Node 3 Data Node 4Data Node 4

• DBA chooses which part of
Primary Key to use as shard key
• Fragment for each row decided
by hashing the sharding key
49
1773467253 chat xxx
6257346892 chat xxx
Shard Key

• Data automatically rearranged to use new capacity
• Designed to be a slow background process not impacting real-time
performance.
50
On-line Scaling and Elasticity
Re-partioning

• Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed
searches
• MySQL Cluster 7.4 DBT2 BM
– 2.5M SQL Statements/Second
52
MySQL Cluster 7.4 SQL Performance
2.5M SQL Statements/Second
-
500,000
1,000,000
1,500,000
2,000,000
2,500,000
3,000,000
2 4 6 8 10 12 14 16
SQLStatements/sec
DataNodes
DBT2SQLStatementsperSecond

• Memory optimized tables
– Durable
– Mix with disk-based tables
• Massively concurrent OLTP
• Distributed Joins for analytics
• Parallel table scans for non-indexed
searches
• MySQL Cluster 7.4 FlexAsych
– 200M NoSQL Reads/Second
53
MySQL Cluster 7.4 NoSQL Performance
200 Million NoSQL Reads/Second
-
50,000,000
100,000,000
150,000,000
200,000,000
250,000,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
Readspersecond
DataNodes
FlexAsyncReads

MySQL NDB Cluster 7.5
• Read Optimized Tables
• Fully Replicated
Capacity and
Scale Out
• MySQL 5.7
• JSON Data Type
• Generated Columns
• Records-Per-Key
Optimization
Improved SQL
• Improved Reporting
• Improved Logging
• Improved Visibility
• Improved Restore
Management
GA
http://mikaelronstrom.blogspot.fr/2016/10/mysql-cluster-75-is-ga-best-cluster.html

• Reading from backup allows to
read from any copy
• Previously all reads were directed
towards the primary fragment only
• Ideal for read-mostly tables
55
MySQL NDB 7.5: Reading from backup
http://mikaelronstrom.blogspot.fr/2016/10/read-any-replica-in-mysql-cluster-75.html

MySQL NDB Cluster 7.5: Fully replicated
• Fully replicated allows a table to be read/written locally on any node
• Ideal for static data, faster join performance
56

Next Steps
Learn More
• www.mysql.com/cluster
• https://dev.mysql.com/doc/mysql-innodb-cluster/en/
• http://mikaelronstrom.blogspot.fr/
• Authentic MySQL Curriculum: http://oracle.com/education/mysql
Learn More
• www.mysql.com/cluster
• https://dev.mysql.com/doc/mysql-innodb-cluster/en/
• http://mikaelronstrom.blogspot.fr/
• Authentic MySQL Curriculum: http://oracle.com/education/mysql
Try it Out
• http://dev.mysql.com/downloads/cluster/
• https://labs.mysql.com/
Try it Out
• http://dev.mysql.com/downloads/cluster/
• https://labs.mysql.com/
Let us know what you think
• http://mysqlhighavailability.com/
• forums.mysql.com/list.php?25
Let us know what you think
• http://mysqlhighavailability.com/
• forums.mysql.com/list.php?25
57

Thank you!

MySQL Day Paris 2016 - MySQL HA: InnoDB Cluster and NDB Cluster

MySQL Day Paris 2016 - MySQL HA: InnoDB Cluster and NDB Cluster

More Related Content

MySQL Day Paris 2016 - MySQL HA: InnoDB Cluster and NDB Cluster