Start Apache kafka on Linux OS Complete Guide || Configuration of Zookeeper, Broker and Consumer and how do they work?

 

 Hi,


Understand Apache Kafka Concepts, Start Zookeeper, Broker & Consumer



Apache Kafka is a publisher-subscriber-based durable messaging system. A messaging system publishes messages on brokers and on the other hand subscriber can subscribe to those either directly or client-key. 

Durable message system

Apache Kafka is software where topics can be defined or created, applications can add, process, and reprocess records.

Applications connect to this system and transfer a message or data onto the topic. A message can include any kind of information; for example, information about an event that has happened on a website, or an event that is supposed to trigger another event. Another application may connect to the system and subscribe and process or re-process records from a topic. The data sent is stored until a specified retention period has passed.



I am sharing a few videos here, so you can take a basic understanding of the Kafka concepts from scratch.

  • To Setup a platform let's first Install Ubuntu - Linux OS:


  • Setup, Config the apache Kafka



  • Start Zookeeper & Broker 



  • Use Consumer - Groups in Apache Kafka :





Regards,
Vidit - A data scientist into Cloud Data Warehouse









Comments

Post a Comment

Popular posts from this blog

Start Work on Apache Kafka || Understand concepts of apache kafka || Apache kafka configuration || Prerequisites Setup || Zookeeper & Broker

Image
 Hi Guys, I am writing this blog for freshers, learners or the experienced techies who want to start work on apache kafka, this is something we can start from scratch, we can start it from Linux OS, you would need Ubuntu OS to install or if you already have then its great :) Also if you need to install Ubuntu OS you can follow the below video link: Install Ubuntu OS using Virtualbox So once you ready with your OS, you can start work on apache kafka, there are prerequisites below before to start on apache commands: 1. Apache Kafka binaries 2. Install Java  3. Setup data logs path for zookeeper and broker To install apache libraries please use the below link Apache Org Link  and download the below libraries.  if you to open this zip file it has the .sh scripts file for bash to run on Unix like systems and there is one windows folder inside bin which have all .bat file if you are working on windows. Once you complete the download, you can start on apache kafka prerequi...
Read more

Understand Apache Kafka - Install Linux Ubuntu OS using Virtualbox Manager on windows OS 10

Image
 Hi, Here we are to start work on apache kafka, this is just for beginners who really want to understand the concepts of kafka, how we can transfer messages, data-streaming using kafka from one end to another, how data flow in micro partitions, encryption etc. You will understand it completely. To start with apache kafka this is necessary you should have a platform i.e. OS, Linux would be the best for it even if you have windows OS, you can install the ubuntu OS on your windows OS using oracle VM virtual box manager, please follow the below video url to start it, Believe me you can do it 😊 and also you can also get hands-on with linux commands. Install Linux - Ubuntu OS using Oracle Virtualbox Once you install the Ubuntu OS, you would be able to start work on Ubuntu OS and can try few linux commands like ll, ls, mkdir, nano, tar xvf --to unzip files etc. have fun 😍 I will meet you in my next blog, how can you start work on apache kafka and install the pre-requisites for apac...
Read more