How to Create Install And Configure Multinode Apach Kafka Cluster

In Section You Will Learn How to Create Install And Configure Multinode Apach Kafka Cluster

Perform all steps into each and every servers


  1. Physical or Virtual Machine ideally with 4 GB RAM, 2 CPU cores and 20 GB disk space
  2. Linux operating system
  3. JDK 8

We will use three centos 7 servers for kafka configuration in GCP(Google cloud platform)

Installing Apache Kafka

centos7 servers into gcp dashborad

In this demo we are going to install - kafka v2.2 and Zookeeper v3.4

Use below link for kafka and zookeeper

  1. Download - kafka v2.2
  2. Download - Zookeeper v3.4

Update and install

 sudo -i
 yum update -y
 yum install wget -y

Do Network configuration

                 kafka1 kafka2 kafka3

Disable Firewall or Add rule if you want

systemctl status firewalld.service
systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl stop firewalld.service


Please make sure JDK 8 is installed


Create apache kafka home directory

mkdir -p /opt/apche_kafka

Download apache zookeeper tarball


Extract apache zookeeper tarball

tar -xvzf zookeeper-3.4.14.tar.gz


Move zookeeper extracted directory into apache kafka home

mv zookeeper-3.4.14 /opt/apche_kafka/

zookeeper directory list apache kafka

Download apache kafka tarball


Extract apache kafka tarball

tar -xvzf kafka_2.12-2.2.0.tgz


Move apache kafka extracted directory into apache kafka home

mv kafka_2.12-2.2.0 /opt/apche_kafka/


Atlast, Change the ownership


chown -R root:root /opt/apche_kafka


Start Configuration:(Do it into all servers)

Zookeeper Configuration

Create zookeeper data directory


mkdir -p /opt/apche_kafka/zookeeper-3.4.14/zkdata


Goto zookpeer config directory


cd /opt/apche_kafka/zookeeper-3.4.14/conf


create file and copy paste below code and save it








Create myid file for all servers 1,2,3 respectively and .cfg file


echo 1 > /opt/apche_kafka/zookeeper-3.4.14/myid
cp /opt/apche_kafka/zookeeper-3.4.14/conf/zoo_sample.cfg /opt/apche_kafka/zookeeper-3.4.14/conf/zoo.cfg


Kafka Configuration

Create kafka data directory


mkdir -p /opt/apche_kafka/kafka_2.12-2.2.0/kafka-logs


Goto kafka config directory


cd /opt/apche_kafka/kafka_2.12-2.2.0/config


Edit file update below parameters (broker id should be unique)



##broker id should be unique for all brokers 


# Enter the zookeeper quorum details as below,,


Start zookeeper into all servers and check status as well


nohup /opt/apche_kafka/zookeeper-3.4.14/bin/ start /opt/apche_kafka/zookeeper-3.4.14/conf/ &

/opt/apche_kafka/zookeeper-3.4.14/bin/ status
check zookpeer status apache kafka

Start apache kafka into all servers


nohup /opt/apche_kafka/kafka_2.12-2.2.0/bin/  /opt/apche_kafka/kafka_2.12-2.2.0/config/ &
start apache kafka broker
If You Want to Know More Checkout My Youtube Channel