副本复制协议

kafka的副本复制协议与目前公开的算法最相似的是PacificA。注意是相似。青老师跟我讲因为PacificA本身是push的模式,而kafka的消息复制是拉的模式。

There are a rich variety of algorithms in this family including ZooKeeper’s Zab, Raft, and Viewstamped Replication. The most similar academic publication we are aware of to Kafka’s actual implementation is PacificA from Microsoft.

复制过程如下:

sequenceDiagram
    replica-->>leader: req1:挂起500ms没有消息就返回,有消息立即返回
    producer-->>leader: produce m1
    leader-->>replica: resp1:立即给副本返回
    replica-->>leader: req2:副本带上自己的LEO过来,leader据此更新HW
    leader-->>producer: m1 resp:更新HW后返回给生产者消息发送成功
    producer-->>leader: produce m2:第二批消息,流程同上
    leader-->>replica: resp2
    replica-->>leader: req2
     leader-->>producer: m2 resp

上面流程图大致描述了复制过程,关于LEO、HW更新细节,后续章节会进行阐述。