Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo
MySQL Multi-Source Replication
Wagner Bianchi - wagner.bianchi@percona.com
Principal Technical Services Engineer
Max Bubenick - max.bubenick@percona.com
Technical Operations Manager
www.percona.com
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
This is gonna be a multi-source replication walkthrough on MariaDB and Percona Server
www.percona.com
•
•
● enables a replication slave to receive transactions from multiple
sources/masters simultaneously;
● aggregate data from multiple servers - Data Mart/Data Warehouse;
● merge table shards - no auto_increment conflict control;
● can be carefully used connecting many location on one geo-positioned
cluster;
● centralize all data for backup propose.
www.percona.com
www.percona.com
www.percona.com
•
•
● No gtid_mode variable to turn on GTID
● Binary logs has positions and GTIDs
• CHANGE MASTER ‘’ TO
• CHANGE MASTER 'foo' TO
• CHANGE MASTER TO
● MASTER_USE_GTID=CURRENT_POS|SLAVE_POS
•
● Accessed using SHOW ALL SLAVES STATUS;
● SET @@default_master_connection works well;
set @@default_master_connection=''; -- default slave connection
show status like 'Slave_running';
set @@default_master_connection='connection_name01'; -- named connection name
show status like 'Slave_running';
www.percona.com
•
box01
Multi-Source Slave
box02
box03
box04
MariaDB [(none)]> pager egrep "Connection|Gtid"
PAGER set to 'egrep "Connection|Seconds"'
MariaDB [(none)]> show all slaves statusG
Connection_name: box02
Gtid_IO_Pos: 1-1-63,3-3-1
Gtid_Slave_Pos: 1-1-63,3-3-1
Connection_name: box03
Gtid_IO_Pos: 1-1-63,3-3-1
Gtid_Slave_Pos: 1-1-63,3-3-1
Connection_name: box04
Gtid_IO_Pos: 1-1-63,3-3-1
Gtid_Slave_Pos: 1-1-63,3-3-1
3 rows in set (0.00 sec)
On the multi-source slave side:
● Use SHOW ALL SLAVES STATUSG
● Use @@default_master_connection
● Use STOP/START ALL SLAVES
● Use STOP/START SLAVE 'box02'
www.percona.com
#: box01 - multi-source slave
[client]
port=3306
socket=/var/lib/mysql/mysql.sock
[mysqld]
user=mysql
port=3306
socket=/var/lib/mysql/mysql.sock
basedir=/usr
datadir=/var/lib/mysql
read_only=1
#: repl vars
server_id=1
report_host=box01
report_port=3306
report_user=repl
log_bin=mysql-bin
log_bin_index=mysql.index
log_slave_updates=true
binlog_format=ROW
#: verify checksum on master
master_verify_checksum=1
#: gtid vars
gtid_domain_id=1
gtid_ignore_duplicates=ON
gtid_strict_mode=1
#: msr slave parallel mode *
box02.slave_parallel_mode=optimistic
box03.slave_parallel_mode=optimistic
box04.slave_parallel_mode=optimistic
#: other slave variables
slave_parallel_threads=16
slave_domain_parallel_threads=4
slave_parallel_max_queued=512M
slave_net_timeout=15
slave_sql_verify_checksum=1
slave_compressed_protocol=1
#: binary log group commit behavior
#binlog_commit_wait_usec=100000
#binlog_commit_wait_count=20
www.percona.com
#: box01
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 1
@@gtid_domain_id: 1
1 row in set (0.00 sec)
#: box02
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 2
@@gtid_domain_id: 2
1 row in set (0.00 sec)
#: box03
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 3
@@gtid_domain_id: 3
1 row in set (0.00 sec)
#: box04
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 4
@@gtid_domain_id: 4
1 row in set (0.00 sec)
box01
Multi-Source Slave
gtid_domain_id=1
gtid_domain_id=2
gtid_domain_id=3
gtid_domain_id=4
www.percona.com
•
● Make sure the replication user is set on all the servers;
● Make sure all the servers has unique server_id and gtid_domain_id;
•
#: Connection name with box02
MariaDB [(none)]> change master 'box02' to master_host='192.168.0.102',
master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos;
#: Connection name with box03
MariaDB [(none)]> change master 'box03' to master_host='192.168.0.103',
master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos;
#: Connection name with box04
MariaDB [(none)]> change master 'box04' to master_host='192.168.0.104',
master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos;
www.percona.com
•
# start all slaves
MariaDB [(none)]> start all slaves;
Query OK, 0 rows affected, 3 warnings (0.02 sec)
MariaDB [(none)]> show warnings;
+-------+------+-----------------------+
| Level | Code | Message |
+-------+------+-----------------------+
| Note | 1937 | SLAVE 'box04' started |
| Note | 1937 | SLAVE 'box03' started |
| Note | 1937 | SLAVE 'box02' started |
+-------+------+-----------------------+
3 rows in set (0.00 sec)
MariaDB [(none)]> stop all slaves;
Query OK, 0 rows affected, 3 warnings (0.02 sec)
www.percona.com
•
#: setting the @@default_master_connection
MariaDB [(none)]> set @@default_master_connection='box02';
Query OK, 0 rows affected (0.00 sec)
#: showing some box02 Connection name's status replication variables
MariaDB [(none)]> pager egrep "Slave_IO_State|Using_Gtid|Gtid_IO_Pos"
PAGER set to 'egrep "Slave_IO_State|Using_Gtid|Gtid_IO_Pos"'
MariaDB [(none)]> show slave statusG
Slave_IO_State: Waiting for master to send event
Using_Gtid: Current_Pos
Gtid_IO_Pos: 1-1-64,2-2-1,3-3-1
1 row in set (0.00 sec)
#: starting and stopping just box02, as per @@default_master_connection set
MariaDB [(none)]> stop slave;
Query OK, 0 rows affected (0.01 sec)
MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.02 sec)
www.percona.com
•
#: relay logs - one group for each set connection name - host+relay-bin+connection_name
-rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box02.000001
-rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box02.000002
-rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box02.index
-rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box03.000001
-rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box03.000002
-rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box03.index
-rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box04.000001
-rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box04.000002
-rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box04.index
#: master.info - one per set connection name - master-connection_name.info
-rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box02.info
-rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box03.info
-rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box04.info
#: multi-master.info file, listing all set connections names
-rw-rw---- 1 mysql mysql 18 Feb 15 11:00 multi-master.info
[root@maria01 mysql]# cat multi-master.info #: don't edit this file :)
box02
box03
box04
www.percona.com
•
•
•
#: check gtid_current and gtid_slave pos variables
MariaDB [mysql]> select @@gtid_current_pos, @@gtid_slave_posG
*************************** 1. row ***************************
@@gtid_current_pos: 1-1-64,2-2-1,3-3-1
@@gtid_slave_pos: 1-1-64,2-2-1,3-3-1
1 row in set (0.00 sec)
#: check current clave current and slave pos from mysql.gtid_slave_pos table
MariaDB [mysql]> select * from mysql.gtid_slave_pos;
+-----------+--------+-----------+--------+
| domain_id | sub_id | server_id | seq_no |
+-----------+--------+-----------+--------+
| 1 | 16 | 1 | 63 |
| 1 | 20 | 1 | 64 |
| 3 | 1 | 3 | 1 |
+-----------+--------+-----------+--------+
3 rows in set (0.00 sec)
www.percona.com
• @@GTID_CURRENT_POS
•
● CHANGE MASTER 'foo' TO … MASTER_USE_GTID=CURRENT_POS;
● @@GTID_STRICT_MODE=1
#: checking global variable @@gtid_current_pos
MariaDB [(none)]> SELECT @@GTID_CURRENT_POS,@@GTID_STRICT_MODEG
*************************** 1. row ***************************
@@GTID_CURRENT_POS: 1-1-68,2-2-149378,3-3-88622,4-4-98365
@@GTID_STRICT_MODE: 1 -- help keeping binlogs identical across multiple servers
1 row in set (0.00 sec)
www.percona.com
• @@GTID_SLAVE_POS CHANGE MASTER
TO
• CHANGE MASTER 'foo' TO … MASTER_USE_GTID=SLAVE_POS;
•
#: checking global variable @@gtid_slave_pos
MariaDB [(none)]> SELECT @@GTID_SLAVE_POSG
*************************** 1. row ***************************
@@GTID_SLAVE_POS: 1-1-66,2-2-149378,3-3-88622,4-4-98365
1 row in set (0.00 sec)
#: checking global variable @@gtid_slave_pos (can't be set per Connection Name)
MariaDB [(none)]> SET GLOBAL GTID_SLAVE_POS='1-1-66,2-2-149378,3-3-88622,4-4-98365';
Query OK, 0 rows affected, 3 warnings (0.01 sec)
#: SHOULD BE DONE AFTER STOPPING REPLICATION CONNECTION NAMES...
www.percona.com
•
•
#: comparing both @@gtid_current_pos and @@gtid_slave_pos
MariaDB [(none)]> SELECT @@GLOBAL.GTID_CURRENT_POS,@@GLOBAL.GTID_SLAVE_POSG
*************************** 1. row ***************************
@@GLOBAL.GTID_CURRENT_POS: 1-1-71,2-2-149378,3-3-88622,4-4-98365 -- 5 ADDITIONAL TRXs
@@GLOBAL.GTID_SLAVE_POS: 1-1-66,2-2-149378,3-3-88622,4-4-98365
1 row in set (0.00 sec)
#: setting dynamically @@gtid_slave_pos - TRANSACTIONS CANNOT BE REPLAYED
MariaDB [(none)]> SET GLOBAL GTID_SLAVE_POS='1-1-66,2-2-149378,3-3-88622,4-4-98365';
ERROR 1947 (HY000): Specified GTID 1-1-66 conflicts with the binary log which contains
a more recent GTID 1-1-71. If MASTER_GTID_POS=CURRENT_POS is used, the binlog position
will override the new value of @@gtid_slave_pos.
#: command above was embraced by START/STOP ALL SLAVES.
www.percona.com
•
● slave_parallel_threads sets the number of threads will take care
about the updates on relay logs based on their timestamps;
●
●
●
●
●
●
●
www.percona.com
•
#: making a multi-source slave multi-threaded
MariaDB [(none)]> stop all slaves;
Query OK, 0 rows affected, 3 warnings (0.00 sec)
MariaDB [(none)]> set global slave_parallel_threads=12;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> set global slave_domain_parallel_threads=4;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> start all slaves;
Query OK, 0 rows affected, 3 warnings (0.03 sec)
MariaDB [(none)]> select @@slave_parallel_mode,@@slave_parallel_threads,@@slave_domain_parallel_threadsG
*************************** 1. row ***************************
@@slave_parallel_mode: optimistic # will retry transaction in case of parallelism conflicts
@@slave_parallel_threads: 12 # total of threads available for slave to execute relay logs
@@slave_domain_parallel_threads: 4 # minimum # of thread used for a domain_id all time
1 row in set (0.00 sec)
www.percona.com
•
#: making a multi-source slave multi-threaded
MariaDB [(none)]> SELECT ID,TIME,STATE,USER FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='system user';
+----+------+------------------------------------------------------------------+-------------+
| ID | TIME | STATE | USER |
+----+------+------------------------------------------------------------------+-------------+
| 18 | 452 | Waiting for master to send event | system user |
| 17 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user |
| 16 | 452 | Waiting for master to send event | system user |
| 15 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user |
| 14 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user |
| 13 | 452 | Waiting for master to send event | system user |
| 12 | 452 | Waiting for work from SQL thread | system user |
| 11 | 452 | Waiting for work from SQL thread | system user |
| 10 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 9 | 0 | Unlocking tables | system user |
| 8 | 452 | Waiting for work from SQL thread | system user |
| 7 | 452 | Waiting for work from SQL thread | system user |
| 6 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 5 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 4 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 3 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
+----+------+------------------------------------------------------------------+-------------+
16 rows in set (0.07 sec)
www.percona.com
•
●
● binlog_commit_wait_usec=100000
● binlog_commit_wait_count=20
●
#: binary logs for group commit
[root@maria02 mysql]# mysqlbinlog mysql-bin.000017 -vvvv | egrep "cid=353579"
#160325 21:37:27 server id 2 end_log_pos 27101572 GTID 2-2-149349 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27103107 GTID 2-2-149350 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27104646 GTID 2-2-149351 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27106181 GTID 2-2-149352 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27107716 GTID 2-2-149353 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27109251 GTID 2-2-149354 cid=353579 trans
MariaDB [(none)]> show global status where variable_name in
('Binlog_commits','Binlog_group_commits');
+----------------------+-------+
| Variable_name | Value |
+----------------------+-------+
| Binlog_commits | 39681 |
| Binlog_group_commits | 5523 |
+----------------------+-------+
2 rows in set (0.02 sec)
the bigger the difference between the two variables
the bigger the apparent group commit efficiency
www.percona.com
•
•
•
•
•
•
•
•
www.percona.com
•
● One can restart both threads or just SQL_THREAD;
#: adding new schema to box02
box02> set sql_log_bin=0; create database if not exists box02_new; set sql_log_bin=1;
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
#: adding filters on multi-source slave
box01> stop slave 'box02'; set global box02.replicate_ignore_db='box02_new'; start slave 'box02';
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
#: remove filter
box01> stop slave; set global box02.replicate_ignore_db=''; start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.11 sec)
www.percona.com
• sql_slave_skip_counter
● If parallel replication is enabled:
•
● Stop all slaves, turn off the parallel replication and skip_counter;
● Set a new value for @@GLOBAL.GTID_SLAVE_POS;
MariaDB [box02]> set default_master_connection='box02'; set global sql_slave_skip_counter=1;
Query OK, 0 rows affected (0.00 sec)
ERROR 1966 (HY000): When using parallel replication and GTID with multiple replication domains,
@@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip
to after a given GTID position.
MariaDB [box02]> stop all slaves; set @@global.gtid_slave_pos='1-1-71,2-2-149382,3-3-88623,4-4-98365';
Query OK, 0 rows affected, 3 warnings (0.00 sec)
Query OK, 0 rows affected (0.01 sec)
MariaDB [box02]> start all slaves;
Query OK, 0 rows affected, 3 warnings (0.01 sec)
www.percona.com
www.percona.com
●
● gtid_mode=on
●
●
● PERFORMANCE_SCHEMA
● SHOW SLAVE STATUS
● SHOW SLAVE STATUS FOR CHANNEL
mysql> pager egrep "Slave_IO|Channel"
PAGER set to 'egrep "Slave_IO|Channel"'
mysql> show slave statusG
Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Channel_Name: box02
Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Channel_Name: box03
Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Channel_Name: box04
3 rows in set (0.00 sec)
www.percona.com
box01
Multi-Source Slave
box02
box03
box04
#: multi-source slave configuration file
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
report_host=box01
report_port=3306
report_user=repl
#: gtid configuration
server_id=1
gtid_mode=on
log_bin=mysql-bin
log_bin_index=mysql-bin.index
log_slave_updates=true
enforce_gtid_consistency
#: binary logs (GROUP COMMIT)
binlog_group_commit_sync_delay=100000
binlog_group_commit_sync_no_delay_cont=20
#binlog_order_commits=0 # increase parallelism
#: repositories for crash-safe
master_info_repository=TABLE
relay_log_info_repository=TABLE
#: slave configs
slave_parallel_workers=4
slave_parallel_type='LOGICAL_CLOCK'
slave_compressed_protocol=1
slave_pending_jobs_size_max=256M
www.percona.com
•
•
#: Replication Channel for box02
box01> change master to master_host='192.168.0.12', master_user='repl',
master_password='Bi@nchI', master_auto_position=1 for channel 'box02';
#: Replication Channel for box03
box01> change master to master_host='192.168.0.13', master_user='repl',
master_password='Bi@nchI', master_auto_position=1 for channel 'box03';
#: Replication Channel for box04
box01> change master to master_host='192.168.0.14', master_user='repl',
master_password='Bi@nchI', master_auto_position=1 for channel 'box04';
www.percona.com
#: relay logs - one group for each set connection name
-rw-r----- 1 mysql mysql 847 Mar 27 00:51 percona01-relay-bin-box02.000018
-rw-r----- 1 mysql mysql 534 Mar 27 00:51 percona01-relay-bin-box02.000019
-rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box02.index
-rw-r----- 1 mysql mysql 597 Mar 27 00:51 percona01-relay-bin-box03.000017
-rw-r----- 1 mysql mysql 534 Mar 27 00:51 percona01-relay-bin-box03.000018
-rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box03.index
-rw-r----- 1 mysql mysql 550 Mar 27 00:51 percona01-relay-bin-box04.000017
-rw-r----- 1 mysql mysql 487 Mar 27 00:51 percona01-relay-bin-box04.000018
-rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box04.index
#:No master.info file as it needs to be configured with crash-safe, repos as TABLE
mysql> show variables where variable_name in
('master_info_repository','relay_log_info_repository');
+---------------------------+-------+
| Variable_name | Value |
+---------------------------+-------+
| master_info_repository | TABLE |
| relay_log_info_repository | TABLE |
+---------------------------+-------+
2 rows in set (0.00 sec)
•
www.percona.com
•
#: stopping all slaves
mysql> stop slave;
Query OK, 0 rows affected (0.06 sec)
mysql> show slave statusG
Slave_IO_Running: No
Slave_SQL_Running: No
[...snip...]
3 rows in set (0.00 sec)
#: stopping just one specific slave
mysql> stop slave for channel 'box02';
Query OK, 0 rows affected (0.01 sec)
mysql> show slave status for channel 'box02'G
Slave_IO_Running: No
Slave_SQL_Running: No
Slave_SQL_Running_State:
1 row in set (0.00 sec)
www.percona.com
•
#: starting all slaves
mysql> start slave;
Query OK, 0 rows affected (0.06 sec)
mysql> show slave statusG
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
[...snip...]
3 rows in set (0.00 sec)
#: starting just one specific slave
mysql> start slave for channel 'box02';
Query OK, 0 rows affected (0.01 sec)
mysql> show slave status for channel 'box02'G
Slave_IO_Running: No
Slave_SQL_Running: No
Slave_SQL_Running_State:
1 row in set (0.00 sec)
www.percona.com
•
#: show all slaves status
mysql> show slave statusG
Channel_Name: box02
Channel_Name: box03
Channel_Name: box04
3 rows in set (0.00 sec)
#: show slave status for a specific slave
mysql> show slave status for channel 'box02'G
Slave_IO_State: Waiting for master to send event
Retrieved_Gtid_Set: 61be13a1-d574-11e5-83c7-0800274fb806:1-61...
Executed_Gtid_Set: 4bd77dee-d572-11e5-b09f-0800274fb806:1-56...
Channel_Name: box02
1 row in set (0.00 sec)
www.percona.com
#: PERFORMANCE_SCHEMA replication tables
mysql> show tables from performance_schema like 'replication%';
+---------------------------------------------+
| Tables_in_performance_schema (replication%) |
+---------------------------------------------+
| replication_applier_configuration |
| replication_applier_status |
| replication_applier_status_by_coordinator |
| replication_applier_status_by_worker |
| replication_connection_configuration |
| replication_connection_status |
| replication_group_member_stats |
| replication_group_members |
+---------------------------------------------+
8 rows in set (0.00 sec)
www.percona.com
#: PERFORMANCE_SCHEMA replication tables
mysql> select channel_name,service_state,last_heartbeat_timestamp
-> from performance_schema.replication_connection_statusG
*************************** 1. row ***************************
channel_name: box02
service_state: ON
last_heartbeat_timestamp: 2016-03-26 22:22:39
*************************** 2. row ***************************
channel_name: box03
service_state: ON
last_heartbeat_timestamp: 2016-03-26 22:22:30
*************************** 3. row ***************************
channel_name: box04
service_state: ON
last_heartbeat_timestamp: 2016-03-26 22:22:31
3 rows in set (0.00 sec)
www.percona.com
mysql> select @@slave_parallel_workers, @@slave_parallel_type;
+--------------------------+-----------------------+
| @@slave_parallel_workers | @@slave_parallel_type |
+--------------------------+-----------------------+
| 8 | LOGICAL_CLOCK |
+--------------------------+-----------------------+
1 row in set (0.00 sec)
#: PERFORMANCE_SCHEMA, checking coordinators state (multi-threaded slaves)
mysql> select channel_name, service_state
-> from performance_schema.replication_applier_status_by_coordinatorG
*************************** 1. row ***************************
channel_name: box02
service_state: ON
*************************** 2. row ***************************
channel_name: box03
service_state: ON
*************************** 3. row ***************************
channel_name: box04
service_state: ON
3 rows in set (0.00 sec)
www.percona.com
#: PERFORMANCE_SCHEMA, checking coordinators state (multi-threaded slaves)
mysql> select channel_name,thread_id,service_state,last_seen_transaction
-> from performance_schema.replication_applier_status_by_worker
-> where last_seen_transaction<>''G
*************************** 1. row ***************************
channel_name: box03
thread_id: 71
service_state: ON
last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2536
*************************** 2. row ***************************
channel_name: box03
thread_id: 72
service_state: ON
last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2537
*************************** 3. row ***************************
channel_name: box03
thread_id: 73
service_state: ON
last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2538
...
www.percona.com
• CHANGE REPLICATION FILTER
#: adding new schema to box02
box02> set sql_log_bin=0; create database if not exists box02_new; set sql_log_bin=1;
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
#: adding filters on multi-source slave
mysql> stop slave; change replication filter replicate_ignore_db=(box02_new); start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.11 sec)
#: remove filter
mysql> stop slave; change replication filter replicate_ignore_db=(); start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.11 sec)
www.percona.com
•
• REPLICATE_DO_DB
• REPLICATE_IGNORE_DB
• REPLICATE_DO_TABLE
• REPLICATE_IGNORE_TABLE
• REPLICATE_WILD_DO_TABLE
• REPLICATE_WILD_IGNORE_TABLE
• REPLICATE_REWRITE_DB
•
•
● http://bugs.mysql.com/bug.php?id=80843
www.percona.com
#: rewriting updates on db A to db B
box01> stop slave; change replication filter replicate_rewrite_db=((box02,box03)); start slave;
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.07 sec)
#: writing some data to A to be routed to B
box02> insert into box02.t1 set i=10;
Query OK, 1 row affected (1.01 sec)
#: checking data on A
box01> select * from box02.t1;
Empty set (0.00 sec)
#: checking data on B
box01> select * from box03.t1G
*************************** 1. row ***************************
i: 10
1 row in set (0.00 sec)
#: remove filter
box01> stop slave; change replication filter replicate_rewrite_db=(); start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.12 sec)
● REPLICATE_REWRITE_DB
www.percona.com
•
●
●
box01> SELECT @@HOSTNAME,ID,USER,STATE,TIME,INFO
-> FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='system user';
+------------+----+-------------+--------------------------------------------------------+------+------+
| @@HOSTNAME | ID | USER | STATE | TIME | INFO |
+------------+----+-------------+--------------------------------------------------------+------+------+
| box01 | 32 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 33 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 34 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 35 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 36 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 25 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 26 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 27 | system user | Waiting for master to send event | 721 | NULL |
[...snip...]
+------------+----+-------------+--------------------------------------------------------+------+------+
30 rows in set (0.00 sec)
www.percona.com
•
UUID()
mysql> select * from performance_schema.replication_applier_status_by_workerG
*************************** 1. row ***************************
CHANNEL_NAME: box02
WORKER_ID: 1
THREAD_ID: NULL
SERVICE_STATE: OFF
LAST_SEEN_TRANSACTION: fa11b361-d572-11e5-b63e-0800274fb806:66
LAST_ERROR_NUMBER: 1062
LAST_ERROR_MESSAGE: Worker 0 failed executing transaction 'fa11b361-d572-11e5-b63e-
0800274fb806:66' at master log mysql-bin.000008, end_log_pos 793; Could not execute Write_rows event on table
box02.t1; Duplicate entry '1' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the
event's master log mysql-bin.000008, end_log_pos 793
LAST_ERROR_TIMESTAMP: 2016-04-21 00:18:14
mysql> stop slave for channel 'box02'; set gtid_next='fa11b361-d572-11e5-b63e-0800274fb806:66'; begin;
commit; set gtid_next=automatic; start slave for channel 'box02';
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (1.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.02 sec)
www.percona.com
Feature MariaDB MySQL 5.7
Multi-Source Slave
Creation
CHANGE MASTER 'name' TO...; CHANGE MASTER TO … FOR CHANNEL 'name';
Parallel Threads slave_parallel_mode=optimistic
slave_parallel_threads=16 # ALL
slave_domain_parallel_threads=4
slave_parallel_max_queued=512M
slave_parallel_workers=4 # per RC
slave_parallel_type='LOGICAL_CLOCK'
slave_pending_jobs_size_max=256M
Replication Filters set global box02.
replicate_ignore_db='foo';
CHANGE REPLICATION FILTER ...
Skip Replication Errors SET GLOBAL GTID_SLAVE_POS='1-1-66';
SET GLOBAL
slave_exec_mode='IDEMPOTENT';
SET GLOBAL sql_slave_skip_counter=1;
SET GLOBAL slave_exec_mode='IDEMPOTENT';
SET GTID_NEXT='UUID:TRX_ID'
SET GLOBAL sql_slave_skip_counter=1; start
slave for channel 'xxxx'; (NO GTID)
Possible number of
sources/masters
You can for now only have 64 masters
256 replication channels for any combination
of hostname and port
www.percona.com

More Related Content

MySQL Multi-Source Replication for PL2016

  • 1. MySQL Multi-Source Replication Wagner Bianchi - wagner.bianchi@percona.com Principal Technical Services Engineer Max Bubenick - max.bubenick@percona.com Technical Operations Manager
  • 2. www.percona.com ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● This is gonna be a multi-source replication walkthrough on MariaDB and Percona Server
  • 3. www.percona.com • • ● enables a replication slave to receive transactions from multiple sources/masters simultaneously; ● aggregate data from multiple servers - Data Mart/Data Warehouse; ● merge table shards - no auto_increment conflict control; ● can be carefully used connecting many location on one geo-positioned cluster; ● centralize all data for backup propose.
  • 6. www.percona.com • • ● No gtid_mode variable to turn on GTID ● Binary logs has positions and GTIDs • CHANGE MASTER ‘’ TO • CHANGE MASTER 'foo' TO • CHANGE MASTER TO ● MASTER_USE_GTID=CURRENT_POS|SLAVE_POS • ● Accessed using SHOW ALL SLAVES STATUS; ● SET @@default_master_connection works well; set @@default_master_connection=''; -- default slave connection show status like 'Slave_running'; set @@default_master_connection='connection_name01'; -- named connection name show status like 'Slave_running';
  • 7. www.percona.com • box01 Multi-Source Slave box02 box03 box04 MariaDB [(none)]> pager egrep "Connection|Gtid" PAGER set to 'egrep "Connection|Seconds"' MariaDB [(none)]> show all slaves statusG Connection_name: box02 Gtid_IO_Pos: 1-1-63,3-3-1 Gtid_Slave_Pos: 1-1-63,3-3-1 Connection_name: box03 Gtid_IO_Pos: 1-1-63,3-3-1 Gtid_Slave_Pos: 1-1-63,3-3-1 Connection_name: box04 Gtid_IO_Pos: 1-1-63,3-3-1 Gtid_Slave_Pos: 1-1-63,3-3-1 3 rows in set (0.00 sec) On the multi-source slave side: ● Use SHOW ALL SLAVES STATUSG ● Use @@default_master_connection ● Use STOP/START ALL SLAVES ● Use STOP/START SLAVE 'box02'
  • 8. www.percona.com #: box01 - multi-source slave [client] port=3306 socket=/var/lib/mysql/mysql.sock [mysqld] user=mysql port=3306 socket=/var/lib/mysql/mysql.sock basedir=/usr datadir=/var/lib/mysql read_only=1 #: repl vars server_id=1 report_host=box01 report_port=3306 report_user=repl log_bin=mysql-bin log_bin_index=mysql.index log_slave_updates=true binlog_format=ROW #: verify checksum on master master_verify_checksum=1 #: gtid vars gtid_domain_id=1 gtid_ignore_duplicates=ON gtid_strict_mode=1 #: msr slave parallel mode * box02.slave_parallel_mode=optimistic box03.slave_parallel_mode=optimistic box04.slave_parallel_mode=optimistic #: other slave variables slave_parallel_threads=16 slave_domain_parallel_threads=4 slave_parallel_max_queued=512M slave_net_timeout=15 slave_sql_verify_checksum=1 slave_compressed_protocol=1 #: binary log group commit behavior #binlog_commit_wait_usec=100000 #binlog_commit_wait_count=20
  • 9. www.percona.com #: box01 MariaDB [(none)]> select @@server_id,@@gtid_domain_idG *************** 1. row *************************** @@server_id: 1 @@gtid_domain_id: 1 1 row in set (0.00 sec) #: box02 MariaDB [(none)]> select @@server_id,@@gtid_domain_idG *************** 1. row *************************** @@server_id: 2 @@gtid_domain_id: 2 1 row in set (0.00 sec) #: box03 MariaDB [(none)]> select @@server_id,@@gtid_domain_idG *************** 1. row *************************** @@server_id: 3 @@gtid_domain_id: 3 1 row in set (0.00 sec) #: box04 MariaDB [(none)]> select @@server_id,@@gtid_domain_idG *************** 1. row *************************** @@server_id: 4 @@gtid_domain_id: 4 1 row in set (0.00 sec) box01 Multi-Source Slave gtid_domain_id=1 gtid_domain_id=2 gtid_domain_id=3 gtid_domain_id=4
  • 10. www.percona.com • ● Make sure the replication user is set on all the servers; ● Make sure all the servers has unique server_id and gtid_domain_id; • #: Connection name with box02 MariaDB [(none)]> change master 'box02' to master_host='192.168.0.102', master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos; #: Connection name with box03 MariaDB [(none)]> change master 'box03' to master_host='192.168.0.103', master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos; #: Connection name with box04 MariaDB [(none)]> change master 'box04' to master_host='192.168.0.104', master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos;
  • 11. www.percona.com • # start all slaves MariaDB [(none)]> start all slaves; Query OK, 0 rows affected, 3 warnings (0.02 sec) MariaDB [(none)]> show warnings; +-------+------+-----------------------+ | Level | Code | Message | +-------+------+-----------------------+ | Note | 1937 | SLAVE 'box04' started | | Note | 1937 | SLAVE 'box03' started | | Note | 1937 | SLAVE 'box02' started | +-------+------+-----------------------+ 3 rows in set (0.00 sec) MariaDB [(none)]> stop all slaves; Query OK, 0 rows affected, 3 warnings (0.02 sec)
  • 12. www.percona.com • #: setting the @@default_master_connection MariaDB [(none)]> set @@default_master_connection='box02'; Query OK, 0 rows affected (0.00 sec) #: showing some box02 Connection name's status replication variables MariaDB [(none)]> pager egrep "Slave_IO_State|Using_Gtid|Gtid_IO_Pos" PAGER set to 'egrep "Slave_IO_State|Using_Gtid|Gtid_IO_Pos"' MariaDB [(none)]> show slave statusG Slave_IO_State: Waiting for master to send event Using_Gtid: Current_Pos Gtid_IO_Pos: 1-1-64,2-2-1,3-3-1 1 row in set (0.00 sec) #: starting and stopping just box02, as per @@default_master_connection set MariaDB [(none)]> stop slave; Query OK, 0 rows affected (0.01 sec) MariaDB [(none)]> start slave; Query OK, 0 rows affected (0.02 sec)
  • 13. www.percona.com • #: relay logs - one group for each set connection name - host+relay-bin+connection_name -rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box02.000001 -rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box02.000002 -rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box02.index -rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box03.000001 -rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box03.000002 -rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box03.index -rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box04.000001 -rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box04.000002 -rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box04.index #: master.info - one per set connection name - master-connection_name.info -rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box02.info -rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box03.info -rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box04.info #: multi-master.info file, listing all set connections names -rw-rw---- 1 mysql mysql 18 Feb 15 11:00 multi-master.info [root@maria01 mysql]# cat multi-master.info #: don't edit this file :) box02 box03 box04
  • 14. www.percona.com • • • #: check gtid_current and gtid_slave pos variables MariaDB [mysql]> select @@gtid_current_pos, @@gtid_slave_posG *************************** 1. row *************************** @@gtid_current_pos: 1-1-64,2-2-1,3-3-1 @@gtid_slave_pos: 1-1-64,2-2-1,3-3-1 1 row in set (0.00 sec) #: check current clave current and slave pos from mysql.gtid_slave_pos table MariaDB [mysql]> select * from mysql.gtid_slave_pos; +-----------+--------+-----------+--------+ | domain_id | sub_id | server_id | seq_no | +-----------+--------+-----------+--------+ | 1 | 16 | 1 | 63 | | 1 | 20 | 1 | 64 | | 3 | 1 | 3 | 1 | +-----------+--------+-----------+--------+ 3 rows in set (0.00 sec)
  • 15. www.percona.com • @@GTID_CURRENT_POS • ● CHANGE MASTER 'foo' TO … MASTER_USE_GTID=CURRENT_POS; ● @@GTID_STRICT_MODE=1 #: checking global variable @@gtid_current_pos MariaDB [(none)]> SELECT @@GTID_CURRENT_POS,@@GTID_STRICT_MODEG *************************** 1. row *************************** @@GTID_CURRENT_POS: 1-1-68,2-2-149378,3-3-88622,4-4-98365 @@GTID_STRICT_MODE: 1 -- help keeping binlogs identical across multiple servers 1 row in set (0.00 sec)
  • 16. www.percona.com • @@GTID_SLAVE_POS CHANGE MASTER TO • CHANGE MASTER 'foo' TO … MASTER_USE_GTID=SLAVE_POS; • #: checking global variable @@gtid_slave_pos MariaDB [(none)]> SELECT @@GTID_SLAVE_POSG *************************** 1. row *************************** @@GTID_SLAVE_POS: 1-1-66,2-2-149378,3-3-88622,4-4-98365 1 row in set (0.00 sec) #: checking global variable @@gtid_slave_pos (can't be set per Connection Name) MariaDB [(none)]> SET GLOBAL GTID_SLAVE_POS='1-1-66,2-2-149378,3-3-88622,4-4-98365'; Query OK, 0 rows affected, 3 warnings (0.01 sec) #: SHOULD BE DONE AFTER STOPPING REPLICATION CONNECTION NAMES...
  • 17. www.percona.com • • #: comparing both @@gtid_current_pos and @@gtid_slave_pos MariaDB [(none)]> SELECT @@GLOBAL.GTID_CURRENT_POS,@@GLOBAL.GTID_SLAVE_POSG *************************** 1. row *************************** @@GLOBAL.GTID_CURRENT_POS: 1-1-71,2-2-149378,3-3-88622,4-4-98365 -- 5 ADDITIONAL TRXs @@GLOBAL.GTID_SLAVE_POS: 1-1-66,2-2-149378,3-3-88622,4-4-98365 1 row in set (0.00 sec) #: setting dynamically @@gtid_slave_pos - TRANSACTIONS CANNOT BE REPLAYED MariaDB [(none)]> SET GLOBAL GTID_SLAVE_POS='1-1-66,2-2-149378,3-3-88622,4-4-98365'; ERROR 1947 (HY000): Specified GTID 1-1-66 conflicts with the binary log which contains a more recent GTID 1-1-71. If MASTER_GTID_POS=CURRENT_POS is used, the binlog position will override the new value of @@gtid_slave_pos. #: command above was embraced by START/STOP ALL SLAVES.
  • 18. www.percona.com • ● slave_parallel_threads sets the number of threads will take care about the updates on relay logs based on their timestamps; ● ● ● ● ● ● ●
  • 19. www.percona.com • #: making a multi-source slave multi-threaded MariaDB [(none)]> stop all slaves; Query OK, 0 rows affected, 3 warnings (0.00 sec) MariaDB [(none)]> set global slave_parallel_threads=12; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> set global slave_domain_parallel_threads=4; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> start all slaves; Query OK, 0 rows affected, 3 warnings (0.03 sec) MariaDB [(none)]> select @@slave_parallel_mode,@@slave_parallel_threads,@@slave_domain_parallel_threadsG *************************** 1. row *************************** @@slave_parallel_mode: optimistic # will retry transaction in case of parallelism conflicts @@slave_parallel_threads: 12 # total of threads available for slave to execute relay logs @@slave_domain_parallel_threads: 4 # minimum # of thread used for a domain_id all time 1 row in set (0.00 sec)
  • 20. www.percona.com • #: making a multi-source slave multi-threaded MariaDB [(none)]> SELECT ID,TIME,STATE,USER FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='system user'; +----+------+------------------------------------------------------------------+-------------+ | ID | TIME | STATE | USER | +----+------+------------------------------------------------------------------+-------------+ | 18 | 452 | Waiting for master to send event | system user | | 17 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user | | 16 | 452 | Waiting for master to send event | system user | | 15 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user | | 14 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user | | 13 | 452 | Waiting for master to send event | system user | | 12 | 452 | Waiting for work from SQL thread | system user | | 11 | 452 | Waiting for work from SQL thread | system user | | 10 | 0 | Update_rows_log_event::ha_update_row(-1) | system user | | 9 | 0 | Unlocking tables | system user | | 8 | 452 | Waiting for work from SQL thread | system user | | 7 | 452 | Waiting for work from SQL thread | system user | | 6 | 0 | Update_rows_log_event::ha_update_row(-1) | system user | | 5 | 0 | Update_rows_log_event::ha_update_row(-1) | system user | | 4 | 0 | Update_rows_log_event::ha_update_row(-1) | system user | | 3 | 0 | Update_rows_log_event::ha_update_row(-1) | system user | +----+------+------------------------------------------------------------------+-------------+ 16 rows in set (0.07 sec)
  • 21. www.percona.com • ● ● binlog_commit_wait_usec=100000 ● binlog_commit_wait_count=20 ● #: binary logs for group commit [root@maria02 mysql]# mysqlbinlog mysql-bin.000017 -vvvv | egrep "cid=353579" #160325 21:37:27 server id 2 end_log_pos 27101572 GTID 2-2-149349 cid=353579 trans #160325 21:37:27 server id 2 end_log_pos 27103107 GTID 2-2-149350 cid=353579 trans #160325 21:37:27 server id 2 end_log_pos 27104646 GTID 2-2-149351 cid=353579 trans #160325 21:37:27 server id 2 end_log_pos 27106181 GTID 2-2-149352 cid=353579 trans #160325 21:37:27 server id 2 end_log_pos 27107716 GTID 2-2-149353 cid=353579 trans #160325 21:37:27 server id 2 end_log_pos 27109251 GTID 2-2-149354 cid=353579 trans MariaDB [(none)]> show global status where variable_name in ('Binlog_commits','Binlog_group_commits'); +----------------------+-------+ | Variable_name | Value | +----------------------+-------+ | Binlog_commits | 39681 | | Binlog_group_commits | 5523 | +----------------------+-------+ 2 rows in set (0.02 sec) the bigger the difference between the two variables the bigger the apparent group commit efficiency
  • 23. www.percona.com • ● One can restart both threads or just SQL_THREAD; #: adding new schema to box02 box02> set sql_log_bin=0; create database if not exists box02_new; set sql_log_bin=1; Query OK, 0 rows affected (0.00 sec) Query OK, 1 row affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) #: adding filters on multi-source slave box01> stop slave 'box02'; set global box02.replicate_ignore_db='box02_new'; start slave 'box02'; Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.00 sec) #: remove filter box01> stop slave; set global box02.replicate_ignore_db=''; start slave; Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.11 sec)
  • 24. www.percona.com • sql_slave_skip_counter ● If parallel replication is enabled: • ● Stop all slaves, turn off the parallel replication and skip_counter; ● Set a new value for @@GLOBAL.GTID_SLAVE_POS; MariaDB [box02]> set default_master_connection='box02'; set global sql_slave_skip_counter=1; Query OK, 0 rows affected (0.00 sec) ERROR 1966 (HY000): When using parallel replication and GTID with multiple replication domains, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position. MariaDB [box02]> stop all slaves; set @@global.gtid_slave_pos='1-1-71,2-2-149382,3-3-88623,4-4-98365'; Query OK, 0 rows affected, 3 warnings (0.00 sec) Query OK, 0 rows affected (0.01 sec) MariaDB [box02]> start all slaves; Query OK, 0 rows affected, 3 warnings (0.01 sec)
  • 26. www.percona.com ● ● gtid_mode=on ● ● ● PERFORMANCE_SCHEMA ● SHOW SLAVE STATUS ● SHOW SLAVE STATUS FOR CHANNEL mysql> pager egrep "Slave_IO|Channel" PAGER set to 'egrep "Slave_IO|Channel"' mysql> show slave statusG Slave_IO_State: Waiting for master to send event Slave_IO_Running: Yes Channel_Name: box02 Slave_IO_State: Waiting for master to send event Slave_IO_Running: Yes Channel_Name: box03 Slave_IO_State: Waiting for master to send event Slave_IO_Running: Yes Channel_Name: box04 3 rows in set (0.00 sec)
  • 27. www.percona.com box01 Multi-Source Slave box02 box03 box04 #: multi-source slave configuration file [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock user=mysql report_host=box01 report_port=3306 report_user=repl #: gtid configuration server_id=1 gtid_mode=on log_bin=mysql-bin log_bin_index=mysql-bin.index log_slave_updates=true enforce_gtid_consistency #: binary logs (GROUP COMMIT) binlog_group_commit_sync_delay=100000 binlog_group_commit_sync_no_delay_cont=20 #binlog_order_commits=0 # increase parallelism #: repositories for crash-safe master_info_repository=TABLE relay_log_info_repository=TABLE #: slave configs slave_parallel_workers=4 slave_parallel_type='LOGICAL_CLOCK' slave_compressed_protocol=1 slave_pending_jobs_size_max=256M
  • 28. www.percona.com • • #: Replication Channel for box02 box01> change master to master_host='192.168.0.12', master_user='repl', master_password='Bi@nchI', master_auto_position=1 for channel 'box02'; #: Replication Channel for box03 box01> change master to master_host='192.168.0.13', master_user='repl', master_password='Bi@nchI', master_auto_position=1 for channel 'box03'; #: Replication Channel for box04 box01> change master to master_host='192.168.0.14', master_user='repl', master_password='Bi@nchI', master_auto_position=1 for channel 'box04';
  • 29. www.percona.com #: relay logs - one group for each set connection name -rw-r----- 1 mysql mysql 847 Mar 27 00:51 percona01-relay-bin-box02.000018 -rw-r----- 1 mysql mysql 534 Mar 27 00:51 percona01-relay-bin-box02.000019 -rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box02.index -rw-r----- 1 mysql mysql 597 Mar 27 00:51 percona01-relay-bin-box03.000017 -rw-r----- 1 mysql mysql 534 Mar 27 00:51 percona01-relay-bin-box03.000018 -rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box03.index -rw-r----- 1 mysql mysql 550 Mar 27 00:51 percona01-relay-bin-box04.000017 -rw-r----- 1 mysql mysql 487 Mar 27 00:51 percona01-relay-bin-box04.000018 -rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box04.index #:No master.info file as it needs to be configured with crash-safe, repos as TABLE mysql> show variables where variable_name in ('master_info_repository','relay_log_info_repository'); +---------------------------+-------+ | Variable_name | Value | +---------------------------+-------+ | master_info_repository | TABLE | | relay_log_info_repository | TABLE | +---------------------------+-------+ 2 rows in set (0.00 sec) •
  • 30. www.percona.com • #: stopping all slaves mysql> stop slave; Query OK, 0 rows affected (0.06 sec) mysql> show slave statusG Slave_IO_Running: No Slave_SQL_Running: No [...snip...] 3 rows in set (0.00 sec) #: stopping just one specific slave mysql> stop slave for channel 'box02'; Query OK, 0 rows affected (0.01 sec) mysql> show slave status for channel 'box02'G Slave_IO_Running: No Slave_SQL_Running: No Slave_SQL_Running_State: 1 row in set (0.00 sec)
  • 31. www.percona.com • #: starting all slaves mysql> start slave; Query OK, 0 rows affected (0.06 sec) mysql> show slave statusG Slave_IO_Running: Yes Slave_SQL_Running: Yes [...snip...] 3 rows in set (0.00 sec) #: starting just one specific slave mysql> start slave for channel 'box02'; Query OK, 0 rows affected (0.01 sec) mysql> show slave status for channel 'box02'G Slave_IO_Running: No Slave_SQL_Running: No Slave_SQL_Running_State: 1 row in set (0.00 sec)
  • 32. www.percona.com • #: show all slaves status mysql> show slave statusG Channel_Name: box02 Channel_Name: box03 Channel_Name: box04 3 rows in set (0.00 sec) #: show slave status for a specific slave mysql> show slave status for channel 'box02'G Slave_IO_State: Waiting for master to send event Retrieved_Gtid_Set: 61be13a1-d574-11e5-83c7-0800274fb806:1-61... Executed_Gtid_Set: 4bd77dee-d572-11e5-b09f-0800274fb806:1-56... Channel_Name: box02 1 row in set (0.00 sec)
  • 33. www.percona.com #: PERFORMANCE_SCHEMA replication tables mysql> show tables from performance_schema like 'replication%'; +---------------------------------------------+ | Tables_in_performance_schema (replication%) | +---------------------------------------------+ | replication_applier_configuration | | replication_applier_status | | replication_applier_status_by_coordinator | | replication_applier_status_by_worker | | replication_connection_configuration | | replication_connection_status | | replication_group_member_stats | | replication_group_members | +---------------------------------------------+ 8 rows in set (0.00 sec)
  • 34. www.percona.com #: PERFORMANCE_SCHEMA replication tables mysql> select channel_name,service_state,last_heartbeat_timestamp -> from performance_schema.replication_connection_statusG *************************** 1. row *************************** channel_name: box02 service_state: ON last_heartbeat_timestamp: 2016-03-26 22:22:39 *************************** 2. row *************************** channel_name: box03 service_state: ON last_heartbeat_timestamp: 2016-03-26 22:22:30 *************************** 3. row *************************** channel_name: box04 service_state: ON last_heartbeat_timestamp: 2016-03-26 22:22:31 3 rows in set (0.00 sec)
  • 35. www.percona.com mysql> select @@slave_parallel_workers, @@slave_parallel_type; +--------------------------+-----------------------+ | @@slave_parallel_workers | @@slave_parallel_type | +--------------------------+-----------------------+ | 8 | LOGICAL_CLOCK | +--------------------------+-----------------------+ 1 row in set (0.00 sec) #: PERFORMANCE_SCHEMA, checking coordinators state (multi-threaded slaves) mysql> select channel_name, service_state -> from performance_schema.replication_applier_status_by_coordinatorG *************************** 1. row *************************** channel_name: box02 service_state: ON *************************** 2. row *************************** channel_name: box03 service_state: ON *************************** 3. row *************************** channel_name: box04 service_state: ON 3 rows in set (0.00 sec)
  • 36. www.percona.com #: PERFORMANCE_SCHEMA, checking coordinators state (multi-threaded slaves) mysql> select channel_name,thread_id,service_state,last_seen_transaction -> from performance_schema.replication_applier_status_by_worker -> where last_seen_transaction<>''G *************************** 1. row *************************** channel_name: box03 thread_id: 71 service_state: ON last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2536 *************************** 2. row *************************** channel_name: box03 thread_id: 72 service_state: ON last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2537 *************************** 3. row *************************** channel_name: box03 thread_id: 73 service_state: ON last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2538 ...
  • 37. www.percona.com • CHANGE REPLICATION FILTER #: adding new schema to box02 box02> set sql_log_bin=0; create database if not exists box02_new; set sql_log_bin=1; Query OK, 0 rows affected (0.00 sec) Query OK, 1 row affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) #: adding filters on multi-source slave mysql> stop slave; change replication filter replicate_ignore_db=(box02_new); start slave; Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.11 sec) #: remove filter mysql> stop slave; change replication filter replicate_ignore_db=(); start slave; Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.11 sec)
  • 38. www.percona.com • • REPLICATE_DO_DB • REPLICATE_IGNORE_DB • REPLICATE_DO_TABLE • REPLICATE_IGNORE_TABLE • REPLICATE_WILD_DO_TABLE • REPLICATE_WILD_IGNORE_TABLE • REPLICATE_REWRITE_DB • • ● http://bugs.mysql.com/bug.php?id=80843
  • 39. www.percona.com #: rewriting updates on db A to db B box01> stop slave; change replication filter replicate_rewrite_db=((box02,box03)); start slave; Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.07 sec) #: writing some data to A to be routed to B box02> insert into box02.t1 set i=10; Query OK, 1 row affected (1.01 sec) #: checking data on A box01> select * from box02.t1; Empty set (0.00 sec) #: checking data on B box01> select * from box03.t1G *************************** 1. row *************************** i: 10 1 row in set (0.00 sec) #: remove filter box01> stop slave; change replication filter replicate_rewrite_db=(); start slave; Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.12 sec) ● REPLICATE_REWRITE_DB
  • 40. www.percona.com • ● ● box01> SELECT @@HOSTNAME,ID,USER,STATE,TIME,INFO -> FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='system user'; +------------+----+-------------+--------------------------------------------------------+------+------+ | @@HOSTNAME | ID | USER | STATE | TIME | INFO | +------------+----+-------------+--------------------------------------------------------+------+------+ | box01 | 32 | system user | Waiting for an event from Coordinator | 721 | NULL | | box01 | 33 | system user | Waiting for an event from Coordinator | 721 | NULL | | box01 | 34 | system user | Waiting for an event from Coordinator | 721 | NULL | | box01 | 35 | system user | Waiting for an event from Coordinator | 721 | NULL | | box01 | 36 | system user | Waiting for an event from Coordinator | 721 | NULL | | box01 | 25 | system user | Waiting for an event from Coordinator | 721 | NULL | | box01 | 26 | system user | Waiting for an event from Coordinator | 721 | NULL | | box01 | 27 | system user | Waiting for master to send event | 721 | NULL | [...snip...] +------------+----+-------------+--------------------------------------------------------+------+------+ 30 rows in set (0.00 sec)
  • 41. www.percona.com • UUID() mysql> select * from performance_schema.replication_applier_status_by_workerG *************************** 1. row *************************** CHANNEL_NAME: box02 WORKER_ID: 1 THREAD_ID: NULL SERVICE_STATE: OFF LAST_SEEN_TRANSACTION: fa11b361-d572-11e5-b63e-0800274fb806:66 LAST_ERROR_NUMBER: 1062 LAST_ERROR_MESSAGE: Worker 0 failed executing transaction 'fa11b361-d572-11e5-b63e- 0800274fb806:66' at master log mysql-bin.000008, end_log_pos 793; Could not execute Write_rows event on table box02.t1; Duplicate entry '1' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000008, end_log_pos 793 LAST_ERROR_TIMESTAMP: 2016-04-21 00:18:14 mysql> stop slave for channel 'box02'; set gtid_next='fa11b361-d572-11e5-b63e-0800274fb806:66'; begin; commit; set gtid_next=automatic; start slave for channel 'box02'; Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (1.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.02 sec)
  • 42. www.percona.com Feature MariaDB MySQL 5.7 Multi-Source Slave Creation CHANGE MASTER 'name' TO...; CHANGE MASTER TO … FOR CHANNEL 'name'; Parallel Threads slave_parallel_mode=optimistic slave_parallel_threads=16 # ALL slave_domain_parallel_threads=4 slave_parallel_max_queued=512M slave_parallel_workers=4 # per RC slave_parallel_type='LOGICAL_CLOCK' slave_pending_jobs_size_max=256M Replication Filters set global box02. replicate_ignore_db='foo'; CHANGE REPLICATION FILTER ... Skip Replication Errors SET GLOBAL GTID_SLAVE_POS='1-1-66'; SET GLOBAL slave_exec_mode='IDEMPOTENT'; SET GLOBAL sql_slave_skip_counter=1; SET GLOBAL slave_exec_mode='IDEMPOTENT'; SET GTID_NEXT='UUID:TRX_ID' SET GLOBAL sql_slave_skip_counter=1; start slave for channel 'xxxx'; (NO GTID) Possible number of sources/masters You can for now only have 64 masters 256 replication channels for any combination of hostname and port