DataOps Barcelona - MySQL HA so easy... that's insane !

MySQL HA so easy ? That's insane !
MySQL 8.0 InnoDB Cluster is a very easy High Availability solution for
MySQL

Frédéric Descamps - MySQL Community Manager - Oracle
4 / 147

Save the date !
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.
5 / 147

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purpose only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied up in
making purchasing decisions. The development, release and timing of any features or
functionality described for Oracle´s product remains at the sole discretion of Oracle.
6 / 147

about me - http://about.me/lefred
Who am I ?
7 / 147

Frédéric Descamps
@lefred
MySQL Evangelist
Hacking MySQL since 3.23
devops believer
living in Belgium 🇧🇪
http://lefred.be

8 / 147

MySQL InnoDB Cluster: vision
"A single product - MySQL - with high availability and scaling features baked in;
providing an integrated end-to-end solution that is easy to use."
9 / 147

MySQL InnoDB Cluster: vision
"A single product - MySQL - with high availability and scaling features baked in;
providing an integrated end-to-end solution that is easy to use."
InnoDB
cluster
Ease-of-Use
Extreme Scale-Out
Out-of-Box Solution
Built-in HA
High Performance
Everything Integrated
10 / 147

One Product: MySQL
All components created together
Tested together
Packaged together
Easy to Use
One client: MySQL Shell
Easy packaging
Integrated orchestration
Homogenous servers
Flexible and Modern
SQL and NoSQL together
Protocol Buffers
Asynchronous API
Developer friendly
Support Read/Write Scale Out
Shared clusters
Federated system of N replica sets
supporting cross shard
operations
Each replica set manages a shard
MySQL InnoDB Cluster: goals
11 / 147

MySQL InnoDB Cluster Architecture
Application
MySQL Connector
MySQL Router
MySQL Shell
InnoDB
cluster
Application
MySQL Connector
MySQL Router
Mp
M
M
12 / 147

Let's start with the demo, shall we ?
13 / 147

14 / 147

15 / 147

16 / 147

Group Replication: heart of MySQL InnoDB
Cluster
17 / 147

Group Replication: heart of MySQL InnoDB
Cluster
18 / 147

MySQL Group Replication
but what is it ?!?
19 / 147

but what is it ?!?
GR is a plugin for MySQL, made by MySQL and packaged with MySQL
20 / 147

but what is it ?!?
GR is an implementation of Replicated Database State Machine theory
21 / 147

but what is it ?!?
GR allows to write on all Group Members (cluster nodes) simultaneously while
retaining consistency
22 / 147

but what is it ?!?
GR implements conflict detection and resolution
23 / 147

but what is it ?!?
GR allows automatic distributed recovery
24 / 147

but what is it ?!?
GR allows automatic distributed recovery
Supported on all MySQL platforms !!
Linux, Windows, Solaris, OSX, FreeBSD
25 / 147

And what does it means for users ?
26 / 147

no longer necessary to handle server fail-over manually or with a complicated script
27 / 147

GR provides fault tolerance
28 / 147

GR enables update-everywhere setups
29 / 147

GR handles crashes, failures, re-connetcs automatically
30 / 147

GR handles crashes, failures, re-connetcs automatically
Allows an easy setup of a MySQL service high available !
31 / 147

Default = Single Primary Mode
By default, MySQL InnoDB Cluster runs in Single Primary Mode.
32 / 147

mysql> show global variables like 'group_replication_single_primary_mode';
+---------------------------------------+-------+
| Variable_name | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON |
+---------------------------------------+-------+
33 / 147

mysql> show global variables like 'group_replication_single_primary_mode';
+---------------------------------------+-------+
| Variable_name | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON |
+---------------------------------------+-------+
In Single Primary Mode, a single member acts as the writable master (PRIMARY) and
the rest of the members act as hot-standbys (SECONDARY).
The group itself coordinates and configures itself automatically to determine which
member will act as the PRIMARY, through a leader election mechanism.
34 / 147

35 / 147

MySQL Router
MySQL Router is lightweight middleware that provides transparent routing between your
application and backend MySQL Servers. It can be used for a wide variety of use cases,
such as providing high availability and scalability by effectively routing database traffic
to appropriate backend MySQL Servers.
36 / 147

MySQL Router
MySQL Router is lightweight middleware that provides transparent routing between your
application and backend MySQL Servers. It can be used for a wide variety of use cases,
such as providing high availability and scalability by effectively routing database traffic
to appropriate backend MySQL Servers.
MySQL Router doesn´t require any specific configuration. It configures itself
automatically (bootstrap) using MySQL InnoDB Cluster´s metadata.
37 / 147

ProxySQL has native support for Group
Replication which makes it a good choice
for advanced users.
ProxySQL
If you need some specific features that are not yet available in MySQL Router, like
transparent R/W splitting, then you can use your software of choice.
We are also collaborating with ProxySQL.
38 / 147

MySQL Shell
The MySQL Shell is an interactive Javascript, Python, or SQL interface supporting
development and administration for the MySQL Server and is a component of the MySQL
Server. You can use the MySQL Shell to perform data queries and updates as well as
various administration operations.
39 / 147

MySQL Shell (2)
The MySQL Shell provides:
40 / 147

MySQL Shell (2)
Both Interactive and Batch operations
41 / 147

MySQL Shell (2)
Document and Relational Models
42 / 147

MySQL Shell (2)
CRUD Document and Relational APIs via scripting
43 / 147

MySQL Shell (2)
Traditional Table, JSON, Tab Separated output results formats
44 / 147

MySQL Shell (2)
MySQL Standard and X Protocols
45 / 147

MySQL Shell (2)
MySQL Standard and X Protocols
and more...
46 / 147

New since MySQL 8.0.11
The Shell improved a lot and uses all the enhancements part of MySQL 8:
47 / 147

set persists
48 / 147

set persists
restart
This makes the remote administration very easy and cloud friendly.
49 / 147

don´t forget, this is a HA solution !
50 / 147

Thank you !
Any Questions ?
share your 💕 for MySQL on social media using @MySQL #DataOpsBarcelona
more at http://lefred.be/content/category/mysql/group-replication/
http://mysqlhighavailability.com/
51 / 147

want to know more?
52 / 147

GR is nice, but how does it work ?
53 / 147

it´s just ...
54 / 147

it´s just ...
55 / 147

it´s just ...
... no, in fact the writesets replication is synchronous and then certification and apply of
the changes are local to each nodes and asynchronous.
56 / 147

it´s just ...
... no, in fact the writesets replication is synchronous and then certification and apply of
the changes are local to each nodes and asynchronous.
not that easy to understand... right ? As a picture is worth a 1000 words, let´s illustrate
this...
57 / 147

58 / 147

59 / 147

60 / 147

61 / 147

62 / 147

63 / 147

64 / 147

65 / 147

66 / 147

67 / 147

68 / 147

69 / 147

MySQL Group Replication (full transaction)
70 / 147

71 / 147

72 / 147

73 / 147

74 / 147

75 / 147

76 / 147

77 / 147

78 / 147

MySQL Group Communication System (GCS)
MySQL Xcom protocol
79 / 147

MySQL Xcom protocol
Replicated Database State Machine
80 / 147

MySQL Xcom protocol
Paxos based protocol
81 / 147

MySQL Xcom protocol
its task: deliver messages across the distributed system:
82 / 147

MySQL Xcom protocol
atomically
83 / 147

MySQL Xcom protocol
atomically
in Total Order
84 / 147

MySQL Xcom protocol
atomically
in Total Order
MySQL Group Replication receives the Ordered 'tickets' from this GCS subsystem.
85 / 147

Group Replication : Total Order Delivery - GTID
86 / 147

87 / 147

88 / 147

89 / 147

90 / 147

91 / 147

92 / 147

93 / 147

94 / 147

95 / 147

96 / 147

97 / 147

98 / 147

99 / 147

100 / 147

101 / 147

102 / 147

103 / 147

104 / 147

Group Replication: return commit
Asynchronous Replication:
105 / 147

Group Replication: return from commit (2)
Semi-Sync Replication:
106 / 147

Group Replication: return from commit (3)
Group Replication:
107 / 147

Does this mean we can have a distant node and
always let it ack later ?
108 / 147

NO!
109 / 147

NO!
There will be always a penalty to pay. For example if you have 3 members, every 3rd write
the system will be affected by the lartency... why ?
110 / 147

NO!
There will be always a penalty to pay. For example if you have 3 members, every 3rd write
the system will be affected by the lartency... why ?
That´s because the system has to wait for the noop (single skip message) from the
“distant” node.
More info: http://lefred.be/content/mysql-group-replication-about-ack-from-majority/
111 / 147

Group Replication : Optimistic Locking
Group Replication uses optimistic locking
112 / 147

during a transaction, local (InnoDB) locking happens
113 / 147

optimistically assumes there will be no conflicts across nodes
(no communication between nodes necessary)
114 / 147

cluster-wide conflict resolution happens only at COMMIT, during certification
115 / 147

cluster-wide conflict resolution happens only at COMMIT, during certification
Let´s first have a look at the traditional locking to compare.
116 / 147

Traditional locking
117 / 147

Traditional locking
118 / 147

Traditional locking
119 / 147

Traditional locking
120 / 147

Traditional locking
121 / 147

Traditional locking
122 / 147

Optimistic Locking
123 / 147

Optimistic Locking
124 / 147

Optimistic Locking
125 / 147

Optimistic Locking
126 / 147

Optimistic Locking
127 / 147

Optimistic Locking
128 / 147

Optimistic Locking
The system returns error 149 as certification failed:
ERROR 1180 (HY000): Got error 149 during COMMIT
129 / 147

Such conflicts happen only when using multi-
primary group !
130 / 147

Certification
Certification is the process that only needs to answer the following unique question:
131 / 147

Certification
can the write (transaction) be applied ?
132 / 147

Certification
based on unapplied earlier transactions
133 / 147

Certification
such conflicts must come for other members/nodes
134 / 147

Certification
happens on every member/node
135 / 147

Certification
should be deterministic on every node
136 / 147

Certification
results are not reported to the group
137 / 147

Certification
pass: enter in the apply queue
138 / 147

Certification
fail: drop the transaction
139 / 147

Certification
serialized by the total order in GCS/XCOM + GTID
140 / 147

Certification
cost is based on trx size (# rows & # keys)
141 / 147

Certification
cost is based on trx size (# rows & # keys)
first committer (and ack from majority) wins rule
142 / 147

Drawbacks of optimistic locking
having a first-committer-wins system means conflicts will more likely happen when
writing on multiple members with:
143 / 147

large transactions
144 / 147

large transactions
long running transactions
145 / 147

large transactions
long running transactions
hotspot records
146 / 147

Thank you !
Any Questions ?
share your 💕 for MySQL on social media using @MySQL #DataOpsBarcelona
more at http://lefred.be/content/category/mysql/group-replication/
http://mysqlhighavailability.com/
147 / 147

DataOps Barcelona - MySQL HA so easy... that's insane !

More Related Content

DataOps Barcelona - MySQL HA so easy... that's insane !