mysql主-主数据复制一致性

时间:2022-09-23 15:12:11

As we know mysql do replication asynchronously. I heard that I need some extra plugins to do
synchronous replication.

众所周知,mysql是异步复制的。我听说我需要一些额外的插件来进行同步复制。

So let us Consider the situation of asynchronous replication: The master writes events to its binary log but does not know whether or when a master2 has retrieved and processed them. With asynchronous replication, if the master1 crashes, transactions that it has committed might not have been transmitted to any master2.

因此,让我们考虑一下异步复制的情况:主程序将事件写到它的二进制日志中,但不知道主程序2是否或何时检索并处理了它们。对于异步复制,如果master1崩溃,那么它所提交的事务可能没有被传输到任何master2。

My question is whether these transactions will finally be replicated to master2 later when master1 starts up again? If it is not, then it is a big inconsistency problem.

我的问题是,当master1再次启动时,这些事务是否最终会被复制到master2 ?如果不是,那么这是一个很大的不一致性问题。

My question is same for master-slave replication and master is down with same situation.

我的问题和主从复制是一样的,主从复制的情况是一样的。

Do I need some special configuration parameter to make it happen automatically?

我是否需要一些特殊的配置参数来使它自动发生?

Or Do I have to manually dump out the data from master1 and import to master2 etc?

还是需要手动从master1中转储数据并导入到master2等?

======

= = = = = =

Update: I probably mis-used the word "crashes" above, I just want to refer the situation that master1 fails to sync the data to others for some time period. This replies (thanks) below cover two cases: real un-recoverable crash due to disk failure for example, or temporarily offline problem due to network problem etc.

更新:我可能错误地使用了上面的“崩溃”一词,我只是想看看master1在某个时间段内无法将数据同步到其他人的情况。下面的回复(谢谢)涵盖了两种情况:例如由于磁盘故障导致的真正不可恢复的崩溃,或者由于网络问题导致的临时脱机问题等等。

2 个解决方案

#1


3  

If the master1 comes back online after an interruption, and the binlogs haven't been lost, then the slave can download any binlogs it missed out on. For this discussion, master2 is a slave.

如果master1在中断之后重新联机,并且没有丢失binlogs,那么从服务器可以下载它错过的任何binlog。在这个讨论中,master2是奴隶。

Of course if master1's crash was severe enough to corrupt or lose its binlogs, you're out of luck.

当然,如果master1的崩溃严重到足以腐化或丢失它的绑定日志,你是不幸运的。

There are also cases where master1 wrote changes to its database but the binlog entry was lost in the crash. To solve this risk you can set sync_binlog=1 so that transactions on master1 require that the respective binlog entry is safe on disk or else the transaction can't complete.

在一些情况下,master1对数据库进行了修改,但是在崩溃时丢失了binlog条目。要解决这个风险,可以设置sync_binlog=1,以便master1上的事务需要各自的binlog条目在磁盘上是安全的,否则事务无法完成。

But syncing binlogs to disk on every COMMIT has some overhead, and many sites decide they'd rather have performance than data safety. It's true that some workloads simply have too high a rate of transactions per second to allow this. I.e. there's an physical upper bound to the number of disk syncs per second. The solution would then need to be to get faster disks (or SSD), or spread the workload over multiple servers.

但是,在每个提交上同步二进制文件都有一些开销,而且许多站点决定他们宁愿使用性能而不是数据安全性。的确,有些工作负载每秒的事务速率太高,因此无法实现这一点。也就是说,每秒磁盘同步的数量有一个物理上界。然后,解决方案需要是获得更快的磁盘(或SSD),或者将工作负载分散到多个服务器上。

I've also seen a reverse situation: the binlog entry is written to the filesystem cache on the master1, and the slave downloads that binlog entry and the slave applies it, but that entry never makes it to disk on the master1 because of the disk was having intermittent failures. So ironically, the slave had processed changes that were never committed to disk on the master!

我还看到了一种相反的情况:binlog条目被写到master1上的文件系统缓存中,从服务器下载binlog条目,然后从服务器应用它,但是这个条目永远不会到达master1上的磁盘,因为磁盘有间歇性故障。所以具有讽刺意味的是,从服务器处理的更改从未提交给主服务器上的磁盘!

You mentioned the possibility of synchronous replication. MySQL doesn't really have this option. The closest they have is Semisynchronous Replication which means that a transaction on master1 isn't allowed to commit until at least one semi-sync slave confirms that it has received the transaction's binlog entry. This means you'll never have a discrepancy like those described above. But this adds some latency to every COMMIT.

您提到了同步复制的可能性。MySQL没有这个选项。它们最接近的是半同步复制,这意味着在master1上的事务直到至少一个半同步从属确认它已经接收到事务的binlog条目之前都不允许提交。这意味着你永远不会有上述的差异。但是这增加了每次提交的延迟。

Why is it "semi"-synchronous? Because the COMMIT on master1 doesn't have to wait for the transaction to be executed on the slave, it only has to wait for the slave to acknowledge receiving the binlog event. The slave can still lag behind.

为什么是“半”同步?因为master1上的提交并不需要等待事务在从服务器上执行,它只需要等待从服务器确认接收到binlog事件。奴隶仍然可以落后。

Comment from @ceejayoz mentions Percona XtraDB Cluster. This works pretty similarly to semisynchronous replication, in that during COMMIT, the writer node must transmit the log for that transaction to all other nodes in the cluster. So the latency is the speed at which the slowest node acknowledges receiving the event.

@ceejayoz的评论提到了Percona XtraDB集群。这与半同步复制非常相似,在提交期间,writer节点必须将该事务的日志传输到集群中的所有其他节点。所以延迟是最慢的节点接收事件的速度。

#2


1  

If master1 crashes, the committed transactions will be available at the logfile, and as soon as it is up again, they'll be delivered to the slave(s), in this case, master2. If, by your configuration, there's a chance that while master1 was down master2 committed an identical primary key value, you'll have an inconsistency problem.

如果master1崩溃,提交的事务将在日志文件中可用,一旦它再次启动,它们将被传递给从事务(在本例中是master2)。如果根据配置,有可能在master1下master2提交了相同的主键值时,出现不一致的问题。

You may prevent this by assigning different primary keys by server — for example, one only writes odd numbers while the other even numbers. Or even a composed primary key that uses the server id.

您可以通过服务器分配不同的主键来防止这种情况——例如,一个只写奇数,而另一个写偶数。甚至是使用服务器id的组合主键。

#1


3  

If the master1 comes back online after an interruption, and the binlogs haven't been lost, then the slave can download any binlogs it missed out on. For this discussion, master2 is a slave.

如果master1在中断之后重新联机,并且没有丢失binlogs,那么从服务器可以下载它错过的任何binlog。在这个讨论中,master2是奴隶。

Of course if master1's crash was severe enough to corrupt or lose its binlogs, you're out of luck.

当然,如果master1的崩溃严重到足以腐化或丢失它的绑定日志,你是不幸运的。

There are also cases where master1 wrote changes to its database but the binlog entry was lost in the crash. To solve this risk you can set sync_binlog=1 so that transactions on master1 require that the respective binlog entry is safe on disk or else the transaction can't complete.

在一些情况下,master1对数据库进行了修改,但是在崩溃时丢失了binlog条目。要解决这个风险,可以设置sync_binlog=1,以便master1上的事务需要各自的binlog条目在磁盘上是安全的,否则事务无法完成。

But syncing binlogs to disk on every COMMIT has some overhead, and many sites decide they'd rather have performance than data safety. It's true that some workloads simply have too high a rate of transactions per second to allow this. I.e. there's an physical upper bound to the number of disk syncs per second. The solution would then need to be to get faster disks (or SSD), or spread the workload over multiple servers.

但是,在每个提交上同步二进制文件都有一些开销,而且许多站点决定他们宁愿使用性能而不是数据安全性。的确,有些工作负载每秒的事务速率太高,因此无法实现这一点。也就是说,每秒磁盘同步的数量有一个物理上界。然后,解决方案需要是获得更快的磁盘(或SSD),或者将工作负载分散到多个服务器上。

I've also seen a reverse situation: the binlog entry is written to the filesystem cache on the master1, and the slave downloads that binlog entry and the slave applies it, but that entry never makes it to disk on the master1 because of the disk was having intermittent failures. So ironically, the slave had processed changes that were never committed to disk on the master!

我还看到了一种相反的情况:binlog条目被写到master1上的文件系统缓存中,从服务器下载binlog条目,然后从服务器应用它,但是这个条目永远不会到达master1上的磁盘,因为磁盘有间歇性故障。所以具有讽刺意味的是,从服务器处理的更改从未提交给主服务器上的磁盘!

You mentioned the possibility of synchronous replication. MySQL doesn't really have this option. The closest they have is Semisynchronous Replication which means that a transaction on master1 isn't allowed to commit until at least one semi-sync slave confirms that it has received the transaction's binlog entry. This means you'll never have a discrepancy like those described above. But this adds some latency to every COMMIT.

您提到了同步复制的可能性。MySQL没有这个选项。它们最接近的是半同步复制,这意味着在master1上的事务直到至少一个半同步从属确认它已经接收到事务的binlog条目之前都不允许提交。这意味着你永远不会有上述的差异。但是这增加了每次提交的延迟。

Why is it "semi"-synchronous? Because the COMMIT on master1 doesn't have to wait for the transaction to be executed on the slave, it only has to wait for the slave to acknowledge receiving the binlog event. The slave can still lag behind.

为什么是“半”同步?因为master1上的提交并不需要等待事务在从服务器上执行,它只需要等待从服务器确认接收到binlog事件。奴隶仍然可以落后。

Comment from @ceejayoz mentions Percona XtraDB Cluster. This works pretty similarly to semisynchronous replication, in that during COMMIT, the writer node must transmit the log for that transaction to all other nodes in the cluster. So the latency is the speed at which the slowest node acknowledges receiving the event.

@ceejayoz的评论提到了Percona XtraDB集群。这与半同步复制非常相似,在提交期间,writer节点必须将该事务的日志传输到集群中的所有其他节点。所以延迟是最慢的节点接收事件的速度。

#2


1  

If master1 crashes, the committed transactions will be available at the logfile, and as soon as it is up again, they'll be delivered to the slave(s), in this case, master2. If, by your configuration, there's a chance that while master1 was down master2 committed an identical primary key value, you'll have an inconsistency problem.

如果master1崩溃,提交的事务将在日志文件中可用,一旦它再次启动,它们将被传递给从事务(在本例中是master2)。如果根据配置,有可能在master1下master2提交了相同的主键值时,出现不一致的问题。

You may prevent this by assigning different primary keys by server — for example, one only writes odd numbers while the other even numbers. Or even a composed primary key that uses the server id.

您可以通过服务器分配不同的主键来防止这种情况——例如,一个只写奇数,而另一个写偶数。甚至是使用服务器id的组合主键。