MySQL高可用架構之MHA（3）

05-21

MySQL高可用架構之MHA（3）

來自專欄運維之路

五、規劃mha

1）創建mha管理用的複製賬號

每台資料庫(master、slave01、slave02)上都要創建賬號，在這裡以其中master為例.。

[root@master ~]# mysql -e "grant all privileges on *.* to mha_rep@172.16.1.% identified by 123456;flush privileges;"[root@master ~]# mysqlmysql> select host,user from mysql.user;

2）在3台主機上(master、slave01和slave02)上分別安裝mha4mysql-node包

安裝完成後會在/usr/local/bin目錄下生成以下腳本文件：

-r-xr-xr-x 1 root root 15498 4月 2 16:04 apply_diff_relay_logs # 識別差異的中繼日誌事件並將其差異的事件應用於其他的slave-r-xr-xr-x 1 root root 4807 4月 2 16:04 filter_mysqlbinlog # 去除不必要的ROLLBACK事件（MHA已不再使用這個工具）-r-xr-xr-x 1 root root 7401 4月 2 16:04 purge_relay_logs # 清除中繼日誌（不會阻塞SQL線程）-r-xr-xr-x 1 root root 7263 4月 2 16:04 save_binary_logs # 保存和複製master的二進位日誌

這裡以master為例，其它主機同理。

[root@master ~]# yum install perl-DBD-MySQL -y[root@master ~]# rpm -ivh https://downloads.mariadb.com/files/MHA/mha4mysql-node-0.54-0.el6.noarch.rpm

3）在manager上安裝mha4mysql-manager和mha4mysql-node包

MHA Manager中主要包括了幾個管理員的命令行工具，例如master_manger，master_master_switch等。MHA Manger也依賴於perl模塊，具體如下：

安裝完成後會在/usr/local/bin目錄下面生成以下腳本文件

-r-xr-xr-x 1 root root 15498 4月 2 15:59 apply_diff_relay_logs # 識別差異的中繼日誌事件並將其差異的事件應用於其他的slave-r-xr-xr-x 1 root root 4807 4月 2 15:59 filter_mysqlbinlog # 去除不必要的ROLLBACK事件（MHA已不再使用這個工具） -r-xr-xr-x 1 root root 1995 4月 2 16:21 masterha_check_repl # 檢查MySQL複製狀況-r-xr-xr-x 1 root root 1779 4月 2 16:21 masterha_check_ssh # 檢查MHA的SSH配置狀況-r-xr-xr-x 1 root root 1865 4月 2 16:21 masterha_check_status # 檢測當前MHA運行狀態-r-xr-xr-x 1 root root 3201 4月 2 16:21 masterha_conf_host # 添加或刪除配置的server信息-r-xr-xr-x 1 root root 2517 4月 2 16:21 masterha_manager # 啟動MHA-r-xr-xr-x 1 root root 2165 4月 2 16:21 masterha_master_monitor # 檢測master是否宕機-r-xr-xr-x 1 root root 2373 4月 2 16:21 masterha_master_switch # 控制故障轉移（自動或者手動）-r-xr-xr-x 1 root root 3749 4月 2 16:21 masterha_secondary_check # -r-xr-xr-x 1 root root 1739 4月 2 16:21 masterha_stop # -r-xr-xr-x 1 root root 7401 4月 2 15:59 purge_relay_logs # 清除中繼日誌（不會阻塞SQL線程）-r-xr-xr-x 1 root root 7263 4月 2 15:59 save_binary_logs # 保存和複製master的二進位日誌

複製相關腳本到/usr/local/bin目錄(軟體包解壓縮後就有了，不是必須，因為這些腳本不完整，需要自己修改，這是軟體開發著留給我們自己發揮的,如果開啟下面的任何一個腳本對應的參數，而對應這裡的腳本又沒有修改，則會報錯，自己被坑的很慘)

[root@manager ~]# cd mha4mysql-manager-0.56/samples/scripts/ # 這是我們下載解壓軟體的目錄[root@manager scripts]# ll總用量 32-rwxr-xr-x 1 root root 3443 1月 8 2012 master_ip_failover #自動切換時vip管理的腳本，不是必須，如果我們使用keepalived的，我們可以自己編寫腳本完成對vip的管理，比如監控mysql，如果mysql異常，我們停止keepalived就行，這樣vip就會自動漂移 -rwxr-xr-x 1 root root 9186 1月 8 2012 master_ip_online_change#在線切換時vip的管理，不是必須，同樣可以可以自行編寫簡單的shell完成-rwxr-xr-x 1 root root 11867 1月 8 2012 power_manager#故障發生後關閉主機的腳本，不是必須-rwxr-xr-x 1 root root 1360 1月 8 2012 send_report#因故障切換後發送報警的腳本，不是必須，可自行編寫簡單的shell完成。[root@manager ~]# cp * /usr/local/bin/

#在manager上安裝mha4mysql-manager和mha4mysql-node包

[root@manager ~]# yum install perl cpan perl-DBD-MySQL perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl-Net-Telnet -y[root@manager ~]# rpm -ivh https://downloads.mariadb.com/files/MHA/mha4mysql-node-0.54-0.el6.noarch.rpm[root@manager ~]# wget https://downloads.mariadb.com/files/MHA/mha4mysql-manager-0.56.tar.gz[root@manager ~]# tar zvxf mha4mysql-manager-0.56.tar.gz [root@manager ~]# cd mha4mysql-manager-0.56[root@manager ~]# perl Makefile.PL [root@manager mha4mysql-manager-0.56]# make && make install[root@manager mha4mysql-manager-0.56]# mkdir -p /usr/local/mha/scripts[root@manager mha4mysql-manager-0.56]# cp samples/conf/app1.cnf /usr/local/mha/mha.cnf[root@manager mha4mysql-manager-0.56]# cp samples/scripts/* /usr/local/mha/scripts/

4）修改manager端mha的配置文件

記得去注釋

[root@manager mha4mysql-manager-0.56]# vim /usr/local/mha/mha.cnf[server default]user=mha_rep #MHA管理mysql的用戶名password=123456 #MHA管理mysql的密碼manager_workdir=/usr/local/mha #MHA的工作目錄manager_log=/usr/local/mha/manager.log #MHA的日誌路徑ssh_user=root #免秘鑰登陸的用戶名repl_user=backup #主從複製賬號，用來在主從之間同步數據repl_password=backupping_interval=1 #ping間隔時間，用來檢查master是否正常 [server1]hostname=172.16.1.241master_binlog_dir=/application/mysql/data/candidate_master=1 #master宕機後，優先啟用這台作為master [server2]hostname=172.16.1.242master_binlog_dir=/application/mysql/data/candidate_master=1 [server3]hostname=172.16.1.243master_binlog_dir=/application/mysql/data/no_master=1

5）檢查ssh是否暢通

注意：所有主機之間必須做SSH免密鑰登錄。否則報錯。研究了兩天。（通過查看MHA的功能實現過程發現）

[root@manager ~]# masterha_check_ssh --conf=/usr/local/mha/mha.cnf Mon Apr 3 21:42:33 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.Mon Apr 3 21:42:33 2017 - [info] Reading application default configurations from /usr/local/mha/mha.cnf..Mon Apr 3 21:42:33 2017 - [info] Reading server configurations from /usr/local/mha/mha.cnf..Mon Apr 3 21:42:33 2017 - [info] Starting SSH connection tests..Mon Apr 3 21:42:33 2017 - [debug] Mon Apr 3 21:42:33 2017 - [debug] Connecting via SSH from root@172.16.1.241(172.16.1.241:22) to root@172.16.1.242(172.16.1.242:22)..Mon Apr 3 21:42:33 2017 - [debug] ok.Mon Apr 3 21:42:33 2017 - [debug] Connecting via SSH from root@172.16.1.241(172.16.1.241:22) to root@172.16.1.243(172.16.1.243:22)..Mon Apr 3 21:42:33 2017 - [debug] ok.Mon Apr 3 21:42:34 2017 - [debug] Mon Apr 3 21:42:34 2017 - [debug] Connecting via SSH from root@172.16.1.243(172.16.1.243:22) to root@172.16.1.241(172.16.1.241:22)..Mon Apr 3 21:42:34 2017 - [debug] ok.Mon Apr 3 21:42:34 2017 - [debug] Connecting via SSH from root@172.16.1.243(172.16.1.243:22) to root@172.16.1.242(172.16.1.242:22)..Mon Apr 3 21:42:34 2017 - [debug] ok.Mon Apr 3 21:42:34 2017 - [debug] Mon Apr 3 21:42:33 2017 - [debug] Connecting via SSH from root@172.16.1.242(172.16.1.242:22) to root@172.16.1.241(172.16.1.241:22)..Mon Apr 3 21:42:33 2017 - [debug] ok.Mon Apr 3 21:42:33 2017 - [debug] Connecting via SSH from root@172.16.1.242(172.16.1.242:22) to root@172.16.1.243(172.16.1.243:22)..Mon Apr 3 21:42:34 2017 - [debug] ok.Mon Apr 3 21:42:34 2017 - [info] All SSH connection tests passed successfully.

#如果得到以上結果，表明主機之間ssh互信是暢通的

6）masterha_check_repl工具檢查mysql主從複製是否成功

注意：slave01 slave02和master確保已經做好主從複製。否則出錯。（研究22個小時）不懂perl 挺麻煩的。

[root@manager ~]# masterha_check_repl --conf=/usr/local/mha/mha.cnf Mon Apr 3 21:44:13 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.Mon Apr 3 21:44:13 2017 - [info] Reading application default configurations from /usr/local/mha/mha.cnf..Mon Apr 3 21:44:13 2017 - [info] Reading server configurations from /usr/local/mha/mha.cnf..Mon Apr 3 21:44:13 2017 - [info] MHA::MasterMonitor version 0.56.Mon Apr 3 21:44:14 2017 - [info] Dead Servers:Mon Apr 3 21:44:14 2017 - [info] Alive Servers:Mon Apr 3 21:44:14 2017 - [info] 172.16.1.241(172.16.1.241:3306)Mon Apr 3 21:44:14 2017 - [info] 172.16.1.242(172.16.1.242:3306)Mon Apr 3 21:44:14 2017 - [info] 172.16.1.243(172.16.1.243:3306)Mon Apr 3 21:44:14 2017 - [info] Alive Slaves:Mon Apr 3 21:44:14 2017 - [info] 172.16.1.242(172.16.1.242:3306) Version=5.5.32-log (oldest major version between slaves) log-bin:enabledMon Apr 3 21:44:14 2017 - [info] Replicating from 172.16.1.241(172.16.1.241:3306)Mon Apr 3 21:44:14 2017 - [info] Primary candidate for the new Master (candidate_master is set)Mon Apr 3 21:44:14 2017 - [info] 172.16.1.243(172.16.1.243:3306) Version=5.5.32-log (oldest major version between slaves) log-bin:enabledMon Apr 3 21:44:14 2017 - [info] Replicating from 172.16.1.241(172.16.1.241:3306)Mon Apr 3 21:44:14 2017 - [info] Not candidate for the new Master (no_master is set)Mon Apr 3 21:44:14 2017 - [info] Current Alive Master: 172.16.1.241(172.16.1.241:3306)Mon Apr 3 21:44:14 2017 - [info] Checking slave configurations..Mon Apr 3 21:44:14 2017 - [info] read_only=1 is not set on slave 172.16.1.242(172.16.1.242:3306).Mon Apr 3 21:44:14 2017 - [warning] relay_log_purge=0 is not set on slave 172.16.1.242(172.16.1.242:3306).Mon Apr 3 21:44:14 2017 - [warning] relay_log_purge=0 is not set on slave 172.16.1.243(172.16.1.243:3306).Mon Apr 3 21:44:14 2017 - [info] Checking replication filtering settings..Mon Apr 3 21:44:14 2017 - [info] binlog_do_db= , binlog_ignore_db= Mon Apr 3 21:44:14 2017 - [info] Replication filtering check ok.Mon Apr 3 21:44:14 2017 - [info] Starting SSH connection tests..Mon Apr 3 21:44:16 2017 - [info] All SSH connection tests passed successfully.Mon Apr 3 21:44:16 2017 - [info] Checking MHA Node version..Mon Apr 3 21:44:16 2017 - [info] Version check ok.Mon Apr 3 21:44:16 2017 - [info] Checking SSH publickey authentication settings on the current master..Mon Apr 3 21:44:16 2017 - [info] HealthCheck: SSH to 172.16.1.241 is reachable.Mon Apr 3 21:44:17 2017 - [info] Master MHA Node version is 0.54.Mon Apr 3 21:44:17 2017 - [info] Checking recovery script configurations on the current master..Mon Apr 3 21:44:17 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/application/mysql/data/ --output_file=/var/tmp/save_binary_logs_test --manager_version=0.56 --start_file=mysql-bin.000007 Mon Apr 3 21:44:17 2017 - [info] Connecting to root@172.16.1.241(172.16.1.241).. Creating /var/tmp if not exists.. ok. Checking output directory is accessible or not.. ok. Binlog found at /application/mysql/data/, up to mysql-bin.000007Mon Apr 3 21:44:17 2017 - [info] Master setting check done.Mon Apr 3 21:44:17 2017 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..Mon Apr 3 21:44:17 2017 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user=mha_rep --slave_host=172.16.1.242 --slave_ip=172.16.1.242 --slave_port=3306 --workdir=/var/tmp --target_version=5.5.32-log --manager_version=0.56 --relay_log_info=/application/mysql/data/relay-log.info --relay_dir=/application/mysql/data/ --slave_pass=xxxMon Apr 3 21:44:17 2017 - [info] Connecting to root@172.16.1.242(172.16.1.242:22).. Checking slave recovery environment settings.. Opening /application/mysql/data/relay-log.info ... ok. Relay log found at /application/mysql/data, up to slave01-relay-bin.000002 Temporary relay log file is /application/mysql/data/slave01-relay-bin.000002 Testing mysql connection and privileges.. done. Testing mysqlbinlog output.. done. Cleaning up test file(s).. done.Mon Apr 3 21:44:17 2017 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user=mha_rep --slave_host=172.16.1.243 --slave_ip=172.16.1.243 --slave_port=3306 --workdir=/var/tmp --target_version=5.5.32-log --manager_version=0.56 --relay_log_info=/application/mysql/data/relay-log.info --relay_dir=/application/mysql/data/ --slave_pass=xxxMon Apr 3 21:44:17 2017 - [info] Connecting to root@172.16.1.243(172.16.1.243:22).. Checking slave recovery environment settings.. Opening /application/mysql/data/relay-log.info ... ok. Relay log found at /application/mysql/data, up to slave02-relay-bin.000002 Temporary relay log file is /application/mysql/data/slave02-relay-bin.000002 Testing mysql connection and privileges.. done. Testing mysqlbinlog output.. done. Cleaning up test file(s).. done.Mon Apr 3 21:44:18 2017 - [info] Slaves settings check done.Mon Apr 3 21:44:18 2017 - [info] 172.16.1.241 (current master) +--172.16.1.242 +--172.16.1.243Mon Apr 3 21:44:18 2017 - [info] Checking replication health on 172.16.1.242..Mon Apr 3 21:44:18 2017 - [info] ok.Mon Apr 3 21:44:18 2017 - [info] Checking replication health on 172.16.1.243..Mon Apr 3 21:44:18 2017 - [info] ok.Mon Apr 3 21:44:18 2017 - [warning] master_ip_failover_script is not defined.Mon Apr 3 21:44:18 2017 - [warning] shutdown_script is not defined.Mon Apr 3 21:44:18 2017 - [info] Got exit code 0 (Not master dead).MySQL Replication Health is OK.

六、mha實驗模擬

1）在每次做mha實驗的時候，我們都最好先執行如下命令做檢測

[root@manager ~]# masterha_check_ssh --conf=/usr/local/mha/mha.cnf[root@manager ~]# masterha_check_repl --conf=/usr/local/mha/mha.cnf

#確定兩條命令的返回結果都是無異常的，然後啟動mha服務

2）在manager端啟動mha服務並時刻監控日誌文件的輸出變化

[root@manager ~]# nohup masterha_manager --conf=/usr/local/mha/mha.cnf > /tmp/mha_manager.log 2>&1 &[root@manager ~]# ps -ef |grep masterha |grep -v greproot 2840 2470 2 10:53 pts/0 00:00:00 perl /usr/local/bin/masterha_manager --conf=/usr/local/mha/mha.cnf

3）測試master宕機後會自動切換

#測試前查看slave01,slave02的主從同步情況

#slave01

[root@slave01 ~]# mysql -e show slave statusG |egrep Slave_IO_Running:|Slave_SQL_Running: Slave_IO_Running: Yes Slave_SQL_Running: Yes

#slave02

[root@slave02 ~]# mysql -e show slave statusG |egrep Slave_IO_Running:|Slave_SQL_Running: Slave_IO_Running: Yes Slave_SQL_Running: Yes

#停止master的mysql服務

[root@master ~]# service mysqld stopShutting down MySQL (Percona Server)..... SUCCESS!

#manager上查看manager節點日誌

[root@manager ~]# cat /usr/local/mha/manager.log----- Failover Report -----mha: MySQL Master failover 172.16.1.241 to 172.16.1.242 succeededMaster 172.16.1.241 is down!Check MHA Manager logs at manager:/usr/local/mha/manager.log for details.Started automated(non-interactive) failover.The latest slave 172.16.1.242(172.16.1.242:3306) has all relay logs for recovery.Selected 172.16.1.242 as a new master.172.16.1.242: OK: Applying all logs succeeded.172.16.1.243: This host has the latest relay log events.Generating relay diff files from the latest slave succeeded.172.16.1.243: OK: Applying all logs succeeded. Slave started, replicating from 172.16.1.242.172.16.1.242: Resetting slave info succeeded.Master failover to 172.16.1.242(172.16.1.242:3306) completed successfully.

從上面的輸出可以看出整個MHA的切換過程，共包括以下的步驟：

配置文件檢查階段，這個階段會檢查整個集群配置文件配置。
宕機的master處理，這個階段包括虛擬ip摘除操作，主機關機操作（待研究）。
複製dead maste和最新slave相差的relay log，並保存到MHA Manger具體的目錄下。
識別含有最新更新的slave。
應用從binlog伺服器保存的二進位日誌事件（binlog events）。
提升一個slave為新的master進行複製。
使其他的slave連接新的master進行複製。

6）驗證new master（172.16.1.242）

#我們查看slave02的主從同步信息

[root@slave02 ~]# mysql -e show slave statusG |egrep Master_Host|Slave_IO_Running:|Slave_SQL_Running: Master_Host: 172.16.1.242 # 表示已經轉移新的ip Slave_IO_Running: Yes # 表示主從OK Slave_SQL_Running: Yes

4）恢復master服務

#manage刪除故障轉移文件

[root@manager ~]# cat /usr/local/mha/mha.failover.complete [root@manager ~]# rm -rf /usr/local/mha/mha.failover.complete

#master重啟mysql服務

[root@master ~]# service mysqld startStarting MySQL... SUCCESS!

#在manager的日誌文件中找到主從同步的sql語句

[root@manager ~]# grep MASTER_HOST /usr/local/mha/manager.logMon Apr 3 21:50:59 2017 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST=172.16.1.242, MASTER_PORT=3306, MASTER_LOG_FILE=mysql-bin.000016, MASTER_LOG_POS=107, MASTER_USER=backup, MASTER_PASSWORD=xxx;

#在master上啟動主從同步，密碼為backup

master_log_file和master_log_pos參數需要和上面manager的日誌文件中同步的語句參數里的值相同。

mysql> change master to master_host=172.16.1.242,master_user=backup,master_password=backup,master_port=3306,master_log_file=mysql-bin.000016,master_log_pos=107;Query OK, 0 rows affected (1.02 sec)mysql> start slave;Query OK, 0 rows affected (0.00 sec)

#在master和slave02上執行，檢查主從同步是否都正常，這裡以master為例，slave02同理

[root@master ~]# mysql -e show slave statusG |egrep Master_Host|Slave_IO_Running:|Slave_SQL_Running: Master_Host: 172.16.1.242 Slave_IO_Running: Yes Slave_SQL_Running: Yes

5）再次啟動MHA的manager服務，並停止slave01

[root@manager ~]# nohup masterha_manager --conf=/usr/local/mha/mha.cnf > /tmp/mha_manager.log 2>&1 &

#關閉slave01的mysql服務

[root@slave01 ~]# service mysqld stopShutting down MySQL... SUCCESS[root@slave01 ~]#tail -f /usr/local/mha/manager.log ----- Failover Report -----mha: MySQL Master failover 172.16.1.242 to 172.16.1.241 succeededMaster 172.16.1.242 is down!Check MHA Manager logs at manager:/usr/local/mha/manager.log for details.Started automated(non-interactive) failover.The latest slave 172.16.1.241(172.16.1.241:3306) has all relay logs for recovery.Selected 172.16.1.241 as a new master.172.16.1.241: OK: Applying all logs succeeded.172.16.1.243: This host has the latest relay log events.Generating relay diff files from the latest slave succeeded.172.16.1.243: OK: Applying all logs succeeded. Slave started, replicating from 172.16.1.241.172.16.1.241: Resetting slave info succeeded.Master failover to 172.16.1.241(172.16.1.241:3306) completed successfully.

出現故障的快速恢復步驟

[root@slave01 ~]# service mysqld stopShutting down MySQL... SUCCESS[root@manager mha]# tail -f /usr/local/mha/manager.log ----- Failover Report -----mha: MySQL Master failover 172.16.1.242Master 172.16.1.242 is down!Check MHA Manager logs at manager:/usr/local/mha/manager.log for details.Started automated(non-interactive) failover.The latest slave 172.16.1.241(172.16.1.241:3306) has all relay logs for recovery.Got Error so couldnt continue failover from here.#出現無法切換回去，後來經過排查是manager /usr/local/mha/mha.cnf [server1] （比較低級的錯誤，排查很久。不過主要是想跟大家分享出現問題如何恢復到之前的狀態。）hostname=172.16.1.241master_binlog_dir=/application/mysql/data/candidate_master=1r #這裡多加了一個r。#修改完畢hostname=172.16.1.241master_binlog_dir=/application/mysql/data/candidate_master=1 實現文件手動恢復到之前的狀態。#manager[root@manager ~]# rm -rf /usr/local/mha/mha.failover.complete[root@manager ~]# rm -rf /usr/local/mha/mha.failover.error [root@manager ~]# nohup masterha_manager --conf=/usr/local/mha/mha.cnf > /tmp/mha_manager.log 2>&1 &#master[root@master ~]# mysqlmysql> stop slave;mysql> reset slave;mysql> show master statusG*************************** 1. row *************************** File: mysql-bin.000013 Position: 107 Binlog_Do_DB: Binlog_Ignore_DB: 1 row in set (0.00 sec)#slave01[root@slave01 ~]# mysqlmysql> stop slave;mysql> change master to master_host=172.16.1.241,master_user=backup,master_password=backup,master_port=3306,master_log_file=mysql-bin.000013,master_log_pos=107;mysql> start slave;# slave01和slave02恢復之前的狀態。[root@slave01 ~]# mysql -e show slave statusG |egrep Master_Host|Slave_IO_Running:|Slave_SQL_Running: Master_Host: 172.16.1.241 Slave_IO_Running: Yes Slave_SQL_Running: Yes[root@slave02 ~]# mysql -e show slave statusG |egrep Master_Host|Slave_IO_Running:|Slave_SQL_Running: Master_Host: 172.16.1.241 Slave_IO_Running: Yes Slave_SQL_Running: Yes

#manager上查看manager節點日誌

[root@manager ~]# cat /usr/local/mha/manager.log----- Failover Report -----mha: MySQL Master failover 172.16.1.242Master 172.16.1.242 is down!Check MHA Manager logs at manager:/usr/local/mha/manager.log for details.Started automated(non-interactive) failover.The latest slave 172.16.1.241(172.16.1.241:3306) has all relay logs for recovery.Got Error so couldnt continue failover from here.

6）恢復slave01服務

#刪除故障轉移文件

[root@manager ~]# rm -rf /usr/local/mha/mha.failover.complete

#重啟mysql服務

[root@slave01 ~]# service mysqld startStarting MySQL.. SUCCESS!

#在manager的日子文件中找到主從同步的sql語句

[root@manager ~]# grep MASTER_HOST /usr/local/mha/manager.logTue Apr 4 02:47:33 2017 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST=172.16.1.241, MASTER_PORT=3306, MASTER_LOG_FILE=mysql-bin.000015, MASTER_LOG_POS=107, MASTER_USER=backup, MASTER_PASSWORD=xxx;

#在slave01上啟動主從同步，密碼為backup 記得修改MASTER_PASSWORD=xxx 為 MASTER_PASSWORD=bakcup

[root@slave01 ~]# mysqlmysql> stop slavemysql> CHANGE MASTER TO MASTER_HOST=172.16.1.241, MASTER_PORT=3306, MASTER_LOG_FILE=mysql-bin.000015, MASTER_LOG_POS=107, MASTER_USER=backup, MASTER_PASSWORD=backup;Query OK, 0 rows affected (0.39 sec)mysql> start slave;Query OK, 0 rows affected (0.00 sec)

#在slave01和slave02上執行，檢查主從同步是否都正常，

#slave01[root@slave01 ~]# mysql -e show slave statusG |egrep Master_Host|Slave_IO_Running:|Slave_SQL_Running: Master_Host: 172.16.1.241 Slave_IO_Running: Yes Slave_SQL_Running: Yes #slave02 [root@slave02 ~]# mysql -e show slave statusG |egrep Master_Host|Slave_IO_Running:|Slave_SQL_Running: Master_Host: 172.16.1.241 Slave_IO_Running: Yes Slave_SQL_Running: Yes

7）重啟MHA的manager服務

[root@manager ~]# nohup masterha_manager --conf=/usr/local/mha/mha.cnf > /tmp/mha_manager.log 2>&1 &[1] 30389