From 009bee75ab4bc4878a14e915c99be6ae937a0855 Mon Sep 17 00:00:00 2001 From: Calvin Liu <83986057+CalvinConfluent@users.noreply.github.com> Date: Mon, 24 Feb 2025 15:19:18 -0800 Subject: [PATCH] KIP-966 part 1 release doc (#18898) Add notes to explain how ELR and how to manage ELR. Reviewers: Colin P. McCabe --- docs/ops.html | 26 ++++++++++++++++++++++++++ docs/upgrade.html | 7 +++++++ 2 files changed, 33 insertions(+) diff --git a/docs/ops.html b/docs/ops.html index ffaca8bf3b863..3177b49d48722 100644 --- a/docs/ops.html +++ b/docs/ops.html @@ -4281,6 +4281,32 @@

6.12 Eligible Leader Replicas

+ +

Overview

+ +

Starting from Apache Kafka 4.0, Eligible Leader Replicas (KIP-966 Part 1) + is available for the users to an improvement to Kafka replication. As the "strict min ISR" rule has been generally applied, which means the high watermark for the data partition can't advance + if the size of the ISR is smaller than the min ISR(min.insync.replicas), it makes some replicas that are not in the ISR safe to become the leader. The KRaft controller + stores such replicas in the PartitionRecord field called Eligible Leader Replicas. During the leader election, the controller will select the leaders + with the following order:

+ + +

Upgrade & Downgrade

+ +

The ELR is not enabled by default for 4.0. To enable the new protocol on the server, set eligible.leader.replicas.version=1. After that the upgrade, the KRaft + controller will start tracking the ELR.

+

Downgrades are safe to perform by setting eligible.leader.replicas.version=0.

+ +

Tool

+

The ELR fields can be checked through the API DescribeTopicPartitions. The admin client can fetch the ELR info by describing the topics. + Also note that, if min.insync.replicas is updated for a topic, the ELR field will be cleaned. If cluster default min ISR is updated, + all the ELR fields will be cleaned.

+
diff --git a/docs/upgrade.html b/docs/upgrade.html index 4ea791f3a7faa..48b829759e9f7 100644 --- a/docs/upgrade.html +++ b/docs/upgrade.html @@ -81,6 +81,13 @@
Notable changes in 4 When using 4.0 producer clients, the producer epoch is bumped on every transaction to ensure every transaction includes the intended messages and duplicates are not written as part of the next transaction. Downgrading the protocol is safe. For more information check here +
  • + Eligible Leader Replicas (KIP-966 Part 1) + enhances the replication protocol for the Apache Kafka 4.0. Now the KRaft controller keeps track of the data partition replicas that are + not included in ISR but are safe to be elected as leader without data loss. Such replicas are stored in the partition metadata as + the Eligible Leader Replicas(ELR). + For more information check here +
  • Since Apache Kafka 4.0.0, we have added a system property ("org.apache.kafka.sasl.oauthbearer.allowed.urls") to set the allowed URLs as SASL OAUTHBEARER token or jwks endpoints. By default, the value is an empty list.