Apache Kafka Advanced

This course focuses on the use of Apache Kafka for large-scale systems, typically operated in large data centers. We will primarily concentrate on ensuring data availability and data mirroring across data centers.

Objectives:

Currently, we encounter the deployment of Apache Kafka more frequently. Often, it serves as a central element of the entire architecture (which may be based on microservices, for example). However, in practice, it is necessary to ensure data mirroring between multiple Kafka clusters. This ensures high data availability and, last but not least, enables recovery after a potential failure of one Kafka cluster. The tool designed for data mirroring is MirrorMaker 2, and we will practically demonstrate its usage.

Agenda:

• Building a Kafka cluster
• Multiple instances of Zookeepers
• Partition replication among Kafka brokers
• Handling Kafka broker failures
• Interconnecting multiple Kafka clusters
• MirrorMaker tool
• Configuration of the MirrorMaker tool

Target Audience:

• Developers
• Big Data architects
• Project Managers
• Administrators

Prerequisites:

• Basic knowledge of Apache Kafka concepts (topic, partition, producer, consumer, offset)
• Fundamental understanding of computer command-line operations

Technical Requirements: 

To participate in this course, you need:

• PC/Laptop with a good internet connection
• Updated web browser
• Running JRE (Java Runtime Environment) or permissions for its installation