Start Work on Apache Kafka || Understand concepts of apache kafka || Apache kafka configuration || Prerequisites Setup || Zookeeper & Broker

 Hi Guys,


I am writing this blog for freshers, learners or the experienced techies who want to start work on apache kafka, this is something we can start from scratch, we can start it from Linux OS, you would need Ubuntu OS to install or if you already have then its great :)

Also if you need to install Ubuntu OS you can follow the below video link:


So once you ready with your OS, you can start work on apache kafka, there are prerequisites below before to start on apache commands:

1. Apache Kafka binaries

2. Install Java 

3. Setup data logs path for zookeeper and broker


To install apache libraries please use the below link

Apache Org Link and download the below libraries. 


if you to open this zip file it has the .sh scripts file for bash to run on Unix like systems and there is one windows folder inside bin which have all .bat file if you are working on windows.


Once you complete the download, you can start on apache kafka prerequisites and for that I already uploaded a video, please follow this video try the same steps, if you still have any concern please raise it to me in comment box, I will answer to you whenever i will be available 😊.




Thanks,
Vidit - A Data scientist in snowflake, apache kafka and data pipeline 

Comments

  1. UnknownFebruary 3, 2022 at 2:34 AM

    hi buddy,

    I'm trying to write ansible playbook first time, in linux machine trying to run as below:
    - name: Install DataPub
    hosts: windows_servers
    tasks:
    - name: Install kafka
    get_url:
    url: https://dlcdn.apache.org/kafka/3.0.0/kafka_2.13-3.0.0.tgz
    dest: C:\Temp\kafka
    - name: unpack the tar
    unarchive
    src: C:\Temp\kafka\kafka_2.13-3.0.0.tgz
    dest: C:\Temp\kafka
    remote_src: yes

    its gives me an error :
    root@Ansible-Server-01:~/ansible# ansible-playbook -i NewInv.txt DataPub.yaml
    ERROR! conflicting action statements: get_url, dest

    The error appears to be in '/root/ansible/DataPub.yaml': line 4, column 9, but may
    be elsewhere in the file depending on the exact syntax problem.

    The offending line appears to be:

    tasks:
    - name: Install kafka
    ^ here

    ReplyDelete

Post a Comment

Popular posts from this blog

Understand Apache Kafka - Install Linux Ubuntu OS using Virtualbox Manager on windows OS 10

Image
 Hi, Here we are to start work on apache kafka, this is just for beginners who really want to understand the concepts of kafka, how we can transfer messages, data-streaming using kafka from one end to another, how data flow in micro partitions, encryption etc. You will understand it completely. To start with apache kafka this is necessary you should have a platform i.e. OS, Linux would be the best for it even if you have windows OS, you can install the ubuntu OS on your windows OS using oracle VM virtual box manager, please follow the below video url to start it, Believe me you can do it 😊 and also you can also get hands-on with linux commands. Install Linux - Ubuntu OS using Oracle Virtualbox Once you install the Ubuntu OS, you would be able to start work on Ubuntu OS and can try few linux commands like ll, ls, mkdir, nano, tar xvf --to unzip files etc. have fun 😍 I will meet you in my next blog, how can you start work on apache kafka and install the pre-requisites for apac...
Read more

Start Apache kafka on Linux OS Complete Guide || Configuration of Zookeeper, Broker and Consumer and how do they work?

Image
   Hi, Understand Apache Kafka Concepts, Start Zookeeper, Broker & Consumer Apache Kafka is a publisher-subscriber-based durable messaging system. A messaging system publishes messages on brokers and on the other hand subscriber can subscribe to those either directly or client-key.  Apache Kafka is software where topics can be defined or created, applications can add, process, and reprocess records. Applications connect to this system and transfer a message or data onto the topic. A message can include any kind of information; for example, information about an event that has happened on a website, or an event that is supposed to trigger another event. Another application may connect to the system and subscribe and process or re-process records from a topic. The data sent is stored until a specified retention period has passed. I am sharing a few videos here, so you can take a basic understanding of the Kafka concepts from scratch. To Setup a platform let's first Insta...
Read more