状况描述:
今天登录一个MySQL数据库slave节点主机发现/var/lib/mysql下存放大量的mysql-relay-bin文件,最早的文件创建日期甚至是2018年,我记得在slave库同步完master的日志操作记录后,会删除这些文件(默认设置不会删除,我记错了),于是便查看了slave库的状态,发现如下报错:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: *.*.*.*
Master_User: dbsync
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000095
Read_Master_Log_Pos: 869242147
Relay_Log_File: mysqld-relay-bin.000146
Relay_Log_Pos: 871280529
Relay_Master_Log_File: mysql-bin.000075
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB: cdb,cdb_admin
Replicate_Ignore_DB: mysql
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1594
Last_Error: Relay log read failure: Could not parse relay log event entry. The possible reasons are: the master 's binary log is corrupted (you can check this by running ' mysqlbinlog ' on the binary log), the slave' s relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master 's or slave' s MySQL code. If you want to check the master 's binary log or slave' s relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
Skip_Counter: 0
Exec_Master_Log_Pos: 871280384
Relay_Log_Space: 19994786573
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1594
Last_SQL_Error: Relay log read failure: Could not parse relay log event entry. The possible reasons are: the master 's binary log is corrupted (you can check this by running ' mysqlbinlog ' on the binary log), the slave' s relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master 's or slave' s MySQL code. If you want to check the master 's binary log or slave' s relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
1 row in set (0.00 sec)
ERROR:
No query specified
|
原因:
我在master节点上删除了名称为mysql-bin.00007格式的文件,其中包括mysql-bin.000075,因此,slave库找不到该文件,无法同步。
解决办法:
1、在slave库上重新指定同步位置。(不可行)
1
2
3
|
slave stop;
CHANGE MASTER TO MASTER_LOG_FILE= 'mysql-bin.000095' ,MASTER_LOG_POS=869242147; //mysql master节点上mysql-bin.000095的已有位置
slave start;
|
slave节点上show slave status,依然报错,具体的报错内容没有复制下来,只记得errno为1236,Slave_IO_Running进程不运行,Slave_SQL_Running进程运行,大概描述就是某个库的某个表有问题。
在多次尝试指定不同的同步位置(报错的位置,master上mysql-bin-000095刚写过的位置)依然存在该错误。
实际上,表记录已经有问题,就拿描述中提出的那个表来说,slave库存放了约1200条记录,master库则有1900+的记录。除非手工将这些数据补上,否则由于记录操作数据的日志已经丢失(被我删除),是找不到最近的一致的日志操作执行位置的。
2、重做slave库。
由于数据差异太大,而且我觉得不光一张表出现了数据不一样的问题,所以干净点,把从库重做。
1)比对master、slave节点库配置信息,保证一致。(我不知道为什么设置了双主模式,实际上我只有一个实例跑在master节点上啊?)
2)在master、slave节点上查看流量情况(show processlist),保证要重做的slave库上没有业务的流量接入。
3)停止master节点上slave进程。(这个停了以后,我就没开过,不知道有没有问题,待观察)
4)记录master节点上库的日志记录位置,之后备份数据库:
1
2
3
4
5
6
7
8
|
mysql> show master status;
+ ------------------+-----------+-------------------------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+ ------------------+-----------+-------------------------------+------------------+
| mysql-bin.000095 | 871760173 | cdb,cdb_admin | mysql |
+ ------------------+-----------+-------------------------------+------------------+
1 row in set (0.01 sec)
mysqldump -u root -p --databases cdb,cdb_admin > bak.master.sql
|
5)保险起见,备份slave节点库:
1
|
mysqldump -u root -p --databases cdb,cdb_admin > bak.slave.sql
|
6)重做开始:把master库备份文件复制到slave节点上,导入该备份文件
1
|
mysql -u root -p < bak.master.sql
|
7)在slave节点上,重新指定读master日志的位置:
1
2
3
|
slave stop;
CHANGE MASTER TO MASTER_LOG_FILE= 'mysql-bin.000095' ,MASTER_LOG_POS=871760173; //POS为刚才记录的master节点日志记录位置
slave start;
|
8)slave节点上 show slave status;此时Slave_IO_Running,Slave_SQL_Running均运行起来了,刷新slave status,Read_Master_Log_Pos数值也开始增加,重新开始同步了。
总结:
清理文件时,要注意mysql-bin文件在master、slave节点日志读取和写的位置啊!,删之前一定要确认日志位置在master和slave断已被读过,不要乱删,否则搞得slave库无法同步了,就算在slave节点上强行指定master日志读取位置或者跳过该错误,也不排除slave库上数据丢失的可能。
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持服务器之家。
原文链接:https://blog.51cto.com/kotori/2412630