Get started (free)

KRaft mode

Apache Kafka’s KRaft mode replaces Apache ZooKeeper with Kafka’s own built-in consensus mechanism based on the Raft protocol. This simplifies Kafka’s architecture, reducing operational complexity by consolidating cluster metadata management into Kafka itself.

The Stackable Operator for Apache Kafka currently does not support automatic cluster upgrades from Apache ZooKeeper to KRaft.

Overview

  • Introduced: Kafka 2.8.0 (early preview, not production-ready).

  • Matured: Kafka 3.3.x (production-ready, though ZooKeeper is still supported).

  • Default & Recommended: Kafka 3.5+ strongly recommends KRaft for new clusters.

  • Full Replacement: Kafka 4.0.0 (2025) removes ZooKeeper completely.

  • Migration: Tools exist to migrate from ZooKeeper to KRaft, but new deployments should start with KRaft.

Configuration

Enable KRaft mode by adding spec.clusterConfig.metadataManager: KRaft and a Controller role to your cluster manifest as in the example below:

apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
  name: kafka
spec:
  clusterConfig:
    metadataManager: KRaft
  image:
    productVersion: "3.9.2"
  brokers:
    roleGroups:
      default:
        replicas: 1
  controllers:
    roleGroups:
      default:
        replicas: 3
Using spec.controllers is mutually exclusive with spec.clusterConfig.zookeeperConfigMapName. Configuring both would result in an error.

Resources

Each Pod’s resource requirements are the sum of its containers. Controller Pods run an additional quorum-manager sidecar (see Internal operator details) whose resources are fixed and not configurable through spec.controllers.config.resources. Only the kafka container is eligible.

Role Container CPU request CPU limit Memory (request = limit) Storage (logDirs)

Controller

kafka (configurable)

250m

1000m

1Gi

2Gi

Controller

quorum-manager (fixed)

100m

500m

512Mi

 — 

Controller

Pod total

350m

1500m

1.5Gi

2Gi

Broker

kafka (configurable)

250m

1000m

2Gi

2Gi

A minimal KRaft setup consists of 3 Controllers and 3 Brokers:

CPU request CPU limit Memory Storage

3 Controllers

1050m

4500m

4.5Gi

6Gi

3 Brokers

750m

3000m

6Gi

6Gi

Total

1800m

7500m

10.5Gi

12Gi

The Controller replicas should sum up to an odd number for the Raft consensus.

Corresponding to the kafka container values above, the operator uses the following resource defaults:

controllers:
  config:
    resources:
      memory:
        limit: 1Gi
      cpu:
        min: 250m
        max: 1000m
      storage:
        logDirs:
          capacity: 2Gi
brokers:
  config:
    resources:
      memory:
        limit: 2Gi
      cpu:
        min: 250m
        max: 1000m
      storage:
        logDirs:
          capacity: 2Gi

Internal operator details

KRaft mode requires major configuration changes compared to ZooKeeper mode:

  • cluster-id: This is set to the metadata.name of the KafkaCluster resource during initial formatting

  • node.id: This is a calculated integer, hashed from the role and rolegroup and added replica id.

  • process.roles: Will always only be broker or controller. Mixed broker,controller servers are not supported.

  • Each controller pod runs an additional quorum-manager sidecar container that, on startup, admits the pod into the KRaft voter set. On pod restart or termination, the controller is removed from the voter set by the preStop hook on the kafka container.

  • Exactly one controller formats with kafka-storage.sh format --standalone, bootstrapping a single-node quorum by itself. It is the controller with the numerically lowest node.id when the cluster is first reconciled. The operator records this choice in status.kraftBootstrapControllerNodeId of the KafkaCluster and never changes it afterwards, so controller role groups added later (even with lower values for node.id) always join the existing quorum. If the status is lost (e.g. the KafkaCluster is restored from a backup without its status), the operator chooses the bootstrap controller again from the current spec. This is harmless as long as the PersistentVolumeClaims of the controllers are restored too. Every other controller formats with --no-initial-controllers and joins purely through the sidecar’s add-controller call. Brokers always format with --no-initial-controllers too; they are never voters.

  • On controllers, controller.quorum.bootstrap.servers lists the individual pod FQDNs of all controllers, not the role groups' headless Services. A headless Service also resolves to pods that are not ready yet, so a starting controller could resolve it to its own address, and then never find the quorum leader. With one entry per pod, Kafka picks a random entry for every attempt. As a consequence, whenever the number of controller replicas changes, the controller pods are rolled by the restart controller.

  • On brokers, controller.quorum.bootstrap.servers lists a single headless Service, <cluster-name>-controller-headless, which selects the controllers of all role groups. A broker is never part of this Service, so it cannot resolve it to its own address. The Service only resolves to controllers that are ready, i.e. members of the quorum, so a broker never connects to a controller that is still starting. Adding or removing controller role groups or replicas therefore never rolls the brokers.

Kerberos

KRaft clusters support Kerberos (GSSAPI) authentication for clients. Reference an AuthenticationClass with the Kerberos provider as described in Security.

Kerberos is only used on the client-facing listeners of the brokers. The CONTROLLER listener always uses SSL with mutual authentication: brokers and controllers authenticate each other with the certificates of the internalSecretClass, even when Kerberos is enabled. Controllers therefore have no keytab.

Known Issues

  • Automatic migration from Apache ZooKeeper to KRaft is not supported.

  • The quorum is created once by the controller with the lowest node.id using --standalone. If this controller loses its PVC, a new conflicting quorum is created on restart.

  • A Controller that loses its persistent volume is not re-admitted to the voter set automatically, because it returns with a new directory ID while the quorum still lists the old one.

Prevention

  • Use a StorageClass backed by network-attached storage, so that a node failure does not imply volume loss.

  • Do not delete Controller PVCs as part of routine maintenance.

Kraft migration guide

The operator version 26.3.0 adds support for migrating Kafka clusters from ZooKeeper to KRaft mode.

This guide describes the steps required to migrate an existing Kafka cluster managed by the Stackable Kafka operator from ZooKeeper to KRaft mode.

Before starting the migration we recommend to reduce producer/consumer operations to a minimum or even pause them completely if possible to reduce the risk of data loss during the migration.

To make the migration step as clear as possible, we’ll use a complete working example throughout this guide. The example cluster has one ZooKeeper server and three brokers, uses default resources and a single role group, and is secured as follows:

  • Clients authenticate with TLS client certificates, using an AuthenticationClass with the TLS provider (spec.clusterConfig.authentication).

  • Traffic between the Kafka nodes is encrypted with the internalSecretClass kafka-internal-tls, and the client-facing listeners with the tls SecretClass (serverSecretClass).

  • Neither Kerberos nor OPA is used.

The example manifests set no other configuration besides what the migration requires.

The steps in this guide were only validated for the example above. In particular, the interaction of the migration with Kerberos and OPA was not tested. If your cluster uses either of them, check the following points before you start and test the migration on a non-production copy of your cluster first.

Client authentication

The authentication method of the clients does not change during the migration. The CONTROLLER listener is not affected by it: the controllers and brokers always authenticate each other with the certificates of the internalSecretClass, whatever spec.clusterConfig.authentication is set to. Keep spec.clusterConfig.authentication and spec.clusterConfig.tls unchanged in every step, as the example manifests do.

Kerberos

Kerberos is only used on the client-facing listeners of the brokers. The controllers do not use Kerberos and have no keytab, so no Kerberos configuration is needed for them. With Kerberos enabled, the example manifests have not been validated, in particular the broker configOverrides in steps 3 and 4. See Security for the broker-side Kerberos setup.

OPA

When spec.clusterConfig.authorization.opa is set, the operator configures the OPA authorizer on the controllers too, and the Rego rules also authorize the requests between the Kafka nodes (see Security). The controllers you add in step 2 are therefore subject to your Rego rules from the moment they start, and they must be allowed to act as controllers, otherwise the quorum cannot form or the metadata migration fails. The example does not set up OPA, so this has not been validated.

We’ll use Kafka version 3.9.2 for this purpose. This is because this is the last version from the 3.x Kafka series that runs on ZooKeeper mode and is supported by the SDP.

Like any cluster that was set up before the migration was planned, the example cluster does not configure its broker ids. Kafka generated them when the brokers first started, and step 1 below shows how to collect and pin them.

We start by creating a dedicated namespace to work in and deploy the Kafka cluster including ZooKeeper and credentials.

---
apiVersion: v1
kind: Namespace
metadata:
  labels:
    stackable.tech/vendor: Stackable
  name: kraft-migration
---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperCluster
metadata:
  name: simple-zk
  namespace: kraft-migration
spec:
  image:
    productVersion: 3.9.6
    pullPolicy: IfNotPresent
  servers:
    roleGroups:
      default:
        replicas: 1
---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperZnode
metadata:
  name: simple-kafka-znode
  namespace: kraft-migration
spec:
  clusterRef:
    name: simple-zk
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: kafka-internal-tls
spec:
  backend:
    autoTls:
      ca:
        secret:
          name: secret-provisioner-kafka-internal-tls-ca
          namespace: kraft-migration
        autoGenerate: true
---
apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
  name: kafka-client-auth-tls
spec:
  provider:
    tls:
      clientCertSecretClass: kafka-client-auth-secret
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: kafka-client-auth-secret
spec:
  backend:
    autoTls:
      ca:
        secret:
          name: secret-provisioner-tls-kafka-client-ca
          namespace: kraft-migration
        autoGenerate: true
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
  name: simple-kafka
  namespace: kraft-migration
spec:
  image:
    productVersion: 3.9.2
    pullPolicy: IfNotPresent
  clusterConfig:
    metadataManager: ZooKeeper
    authentication:
      - authenticationClass: kafka-client-auth-tls
    tls:
      internalSecretClass: kafka-internal-tls
      serverSecretClass: tls
    zookeeperConfigMapName: simple-kafka-znode
  brokers:
    roleGroups:
      default:
        replicas: 3

Next, from one of the broker pods, we will create a topic called kraft-migration-topic with 3 partitions to verify the migration later.

$ /stackable/kafka/bin/kafka-topics.sh \
--create \
--command-config /stackable/config/client.properties \
--bootstrap-server simple-kafka-broker-default-0-listener-broker.kraft-migration.svc.cluster.local:9093 \
--partitions 3 \
--topic kraft-migration-topic

And - also from one of the broker pods - publish some test messages to it:

$ /stackable/kafka/bin/kafka-producer-perf-test.sh \
--producer.config /stackable/config/client.properties \
--producer-props bootstrap.servers=simple-kafka-broker-default-0-listener-broker.kraft-migration.svc.cluster.local:9093 \
--topic kraft-migration-topic \
--payload-monotonic \
--throughput 1 \
--num-records 100

We now have a working Kafka cluster with ZooKeeper and some test data.

1. Preserve the broker ids

Before KRaft support was added to the operator in SDP 25.11.0, Kafka generated the broker ids and the operator did not know them. In KRaft mode, the id of a broker is its node.id, which must be the same id that is already stored in the broker’s data directory and in the cluster metadata. The operator can only keep the existing ids if you tell it what they are, so this has to be done before anything else.

In this step we will perform the following actions:

  1. Collect the current broker ids from the running brokers.

  2. Store them in a ConfigMap.

  3. Point spec.clusterConfig.brokerIdPodConfigMapName of the KafkaCluster resource to this ConfigMap.

  4. Wait for the brokers to restart and verify that their ids did not change.

Each broker records its id in the meta.properties file in its data directory:

$ kubectl -n kraft-migration exec -c kafka simple-kafka-broker-default-0 -- grep broker.id /stackable/data/topicdata/meta.properties
broker.id=1001

The ids that Kafka generates usually start at 1001, and they are not necessarily in the order of the pod indexes. The following commands collect the id of every broker Pod, of all role groups, and create a ConfigMap with one <pod-name>: <broker-id> entry per Pod:

NAMESPACE=kraft-migration
ARGS=()
for POD in $(kubectl get pods -n $NAMESPACE \
  -l app.kubernetes.io/name=kafka,app.kubernetes.io/instance=simple-kafka,app.kubernetes.io/component=broker \
  -o jsonpath='{.items[*].metadata.name}'); do
  ID=$(kubectl exec -n $NAMESPACE -c kafka $POD -- grep '^broker.id=' /stackable/data/topicdata/meta.properties | cut -d= -f2)
  echo "$POD -> $ID"
  ARGS+=("--from-literal=$POD=$ID")
done
kubectl create configmap broker-ids -n $NAMESPACE "${ARGS[@]}"

The output lists the ids that were found:

simple-kafka-broker-default-0 -> 1001
simple-kafka-broker-default-1 -> 1002
simple-kafka-broker-default-2 -> 1003
configmap/broker-ids created

Check that the ConfigMap contains an entry for every broker Pod before you continue.

Now point the KafkaCluster to the ConfigMap. Create the ConfigMap first, because the broker Pods mount it.

kubectl -n kraft-migration patch kafkacluster simple-kafka --type merge \
  -p '{"spec":{"clusterConfig":{"brokerIdPodConfigMapName":"broker-ids"}}}'

The operator restarts the brokers one after the other. Once they are all ready, each broker must report the same id as before. The broker.id in the configuration file of the broker must match the id in its meta.properties:

$ kubectl -n kraft-migration exec -c kafka simple-kafka-broker-default-0 -- grep broker.id /tmp/broker.properties
broker.id=1001
broker.id.generation.enable=false

The brokerIdPodConfigMapName property and the ConfigMap must be preserved for the lifetime of the cluster, also after the migration is complete. The brokers keep using these ids in KRaft mode, and without the ConfigMap the operator would fall back to generated ids that do not match the ones already stored by the brokers.

The following steps also contain this property in the KafkaCluster manifests.

2. Start Kraft controllers

In this step we will perform the following actions:

  1. Retrieve the current cluster.id as generated by Kafka.

  2. Update the KafkaCluster resource to add spec.controllers property.

  3. Configure the controllers to run in migration mode.

  4. Apply the changes and wait for all cluster pods to become ready.

We can obtain the current cluster.id either by inspecting the ZooKeeper data or from meta.properties file on one of the brokers.

$ kubectl -n kraft-migration exec -c kafka simple-kafka-broker-default-0 -- cat /stackable/data/topicdata/meta.properties | grep cluster.id
cluster.id=MyCya7hbTD-Hay8PgCsCYA

We add this value to the KAFKA_CLUSTER_ID environment variable for both brokers and controllers. The cluster id is generated by Kafka for each cluster, so replace the value MyCya7hbTD-Hay8PgCsCYA in all the example manifests below with your own.

The broker ids were already pinned in step 1 and spec.clusterConfig.brokerIdPodConfigMapName is part of the manifest below.

The KAFKA_CLUSTER_ID environment variable and brokerIdPodConfigMapName must be preserved for the rest of the migration process and the lifetime of the cluster.

The complete example KafkaCluster resource after applying the required changes looks as follows:

---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
  name: simple-kafka
  namespace: kraft-migration
spec:
  image:
    productVersion: 3.9.2
    pullPolicy: IfNotPresent
  clusterConfig:
    metadataManager: ZooKeeper
    authentication:
      - authenticationClass: kafka-client-auth-tls
    tls:
      internalSecretClass: kafka-internal-tls
      serverSecretClass: tls
    zookeeperConfigMapName: simple-kafka-znode
    brokerIdPodConfigMapName: broker-ids
  brokers:
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
    roleGroups:
      default:
        replicas: 3
  controllers:
    roleGroups:
      default:
        replicas: 3
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
    configOverrides:
      controller.properties:
        zookeeper.metadata.migration.enable: "true" # Enable migration mode so the controller can read metadata from ZooKeeper.

We kubectl apply the updated resource and wait for brokers and controllers to become ready.

3. Migrate metadata

In this step we will perform the following actions:

  1. Configure the controller quorum on the brokers.

  2. Enable metadata migration mode on the brokers.

  3. Apply the changes and restart the broker pods.

To start the metadata migration, we need to add the zookeeper.metadata.migration.enable: "true" and controller quorum configuration to the broker configuration.

For this step, the complete example KafkaCluster resource looks as follows:

---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
  name: simple-kafka
  namespace: kraft-migration
spec:
  image:
    productVersion: 3.9.2
    pullPolicy: IfNotPresent
  clusterConfig:
    metadataManager: ZooKeeper
    authentication:
      - authenticationClass: kafka-client-auth-tls
    tls:
      internalSecretClass: kafka-internal-tls
      serverSecretClass: tls
    zookeeperConfigMapName: simple-kafka-znode
    brokerIdPodConfigMapName: broker-ids
  brokers:
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
    roleGroups:
      default:
        replicas: 3
    configOverrides:
      broker.properties:
        inter.broker.protocol.version: "3.9" # - Latest value known to Kafka 3.9.2
        zookeeper.metadata.migration.enable: "true" # - Enable migration mode so the broker can participate in metadata migration.
        controller.listener.names: "CONTROLLER"
        controller.quorum.bootstrap.servers: "simple-kafka-controller-default-0.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-1.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-2.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093"
  controllers:
    roleGroups:
      default:
        replicas: 3
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
    configOverrides:
      controller.properties:
        zookeeper.metadata.migration.enable: "true" # Enable migration mode so the controller can read metadata from ZooKeeper.

The brokers are now restarted automatically due to configuration changes.

Finally we check that metadata migration was successful:

kubectl logs -n kraft-migration simple-kafka-controller-default-0 | grep -i 'completed migration' \
|| kubectl logs -n kraft-migration simple-kafka-controller-default-1 | grep -i 'completed migration' \
|| kubectl logs -n kraft-migration simple-kafka-controller-default-2 | grep -i 'completed migration'

...
[2026-11-19 11:02:00,532] INFO [KRaftMigrationDriver id=2110489703] Completed migration of metadata from ZooKeeper to KRaft. 59 records were generated in 111 ms across 1 batches. The average time spent waiting on a batch was 28.00 ms. The record types were {TOPIC_RECORD=2, PARTITION_RECORD=53, CONFIG_RECORD=3, PRODUCER_IDS_RECORD=1}. The current metadata offset is now 452 with an epoch of 1. Saw 3 brokers in the migrated metadata [1001, 1002, 1003]. (org.apache.kafka.metadata.migration.KRaftMigrationDriver)

The exact counts depend on the topics and brokers of your cluster. Only the controller that leads the quorum logs this message, hence the || chain above.

4. Migrate brokers

This is the last step before fully switching to KRaft mode. In case of unforeseen issues, it is the last step where we can roll back to ZooKeeper mode.

In this step we will perform the following actions:

  1. Remove the migration properties from the previous step on the brokers.

  2. Assign Kraft role properties to brokers.

  3. Apply the changes and restart the broker pods.

We need to preserve the quorum configuration added in the previous step.

For this step, the complete example KafkaCluster resource looks as follows:

---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
  name: simple-kafka
  namespace: kraft-migration
spec:
  image:
    productVersion: 3.9.2
    pullPolicy: IfNotPresent
  clusterConfig:
    metadataManager: ZooKeeper
    authentication:
      - authenticationClass: kafka-client-auth-tls
    tls:
      internalSecretClass: kafka-internal-tls
      serverSecretClass: tls
    zookeeperConfigMapName: simple-kafka-znode
    brokerIdPodConfigMapName: broker-ids
  brokers:
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
    roleGroups:
      default:
        replicas: 3
    configOverrides:
      broker.properties:
        controller.listener.names: "CONTROLLER"
        controller.quorum.bootstrap.servers: "simple-kafka-controller-default-0.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-1.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-2.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093"
        process.roles: "broker"
        node.id: "${env:REPLICA_ID}"
  controllers:
    roleGroups:
      default:
        replicas: 3
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
    configOverrides:
      controller.properties:
        zookeeper.metadata.migration.enable: "true" # Enable migration mode so the controller can read metadata from ZooKeeper.

5. Enable Kraft mode

After this step, the cluster will be fully running in KRaft mode and it cannot be rolled back to ZooKeeper mode anymore.

In this step we will perform the following actions:

  1. Put the cluster in Kraft mode by updating the spec.clusterConfig.metadataManager property.

  2. Remove Kraft quorum configuration from the broker pods.

  3. Remove the ZooKeeper migration flag from the controllers.

  4. Apply the changes and restart all pods.

We need to preserve the KAFKA_CLUSTER_ID environment variable for the rest of the lifetime of this cluster.

The complete example KafkaCluster resource after applying the required changes looks as follows:

---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
  name: simple-kafka
  namespace: kraft-migration
spec:
  image:
    productVersion: 3.9.2
    pullPolicy: IfNotPresent
  clusterConfig:
    metadataManager: KRaft
    authentication:
      - authenticationClass: kafka-client-auth-tls
    tls:
      internalSecretClass: kafka-internal-tls
      serverSecretClass: tls
    brokerIdPodConfigMapName: broker-ids
  brokers:
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
    roleGroups:
      default:
        replicas: 3
    configOverrides:
      broker.properties:
        controller.listener.names: "CONTROLLER"
  controllers:
    roleGroups:
      default:
        replicas: 3
    envOverrides:
      KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"

Verify that the cluster is healthy and consumer/producer operations work as expected.

We can consume the previously produced messages by running the command below on one of the broker pods:

/stackable/kafka/bin/kafka-console-consumer.sh \
--consumer.config /stackable/config/client.properties \
--bootstrap-server simple-kafka-broker-default-0-listener-broker.kraft-migration.svc.cluster.local:9093 \
--topic kraft-migration-topic \
--from-beginning \
--max-messages 100 \
--timeout-ms 30000

The consumer should report Processed a total of 100 messages. We read from all partitions because the producer does not necessarily write to partition 0.

6. Cleanup

Before proceeding with this step please ensure that the Kafka cluster is fully operational in KRaft mode.

In this step we remove the now unused ZooKeeper cluster and related resources.

If the ZooKeeper cluster is also serving other use cases than Kafka you can skip this step.

In our example we can remove the ZooKeeper cluster and the Znode resource as follows:

kubectl delete -n kraft-migration zookeeperznodes simple-kafka-znode
kubectl delete -n kraft-migration zookeeperclusters simple-zk

7. Next steps

After successfully migrating to Kraft mode, consider updating to Kafka version 4 to benefit from the latest features and improvements in KRaft mode.