Apache Kafka
Understand brokers, topics, partitions, offsets, replication, producers, consumers and consumer groups.
Cloudhill · Hands-on Training
Kafka & Real-Time Event Streaming
Build job-ready skills in Apache Kafka, Schema Registry, Kafka Connect, Apache Flink and Confluent Cloud through guided labs and a real-world event-driven project.
Mentor-led learning
Hands-on labs
Capstone project
15 daysStructured learning path
1 hour/dayGuided daily sessions
Hands-onPractical coding labs
1 capstoneEnd-to-end project
Course highlights
Learn the complete event streaming workflow
Start with Kafka fundamentals, then progress to schema governance, data integration, stream processing and a managed cloud platform.
Schema Registry
Work with Avro and schema IDs, compatibility rules, schema versions and safe schema evolution.
Kafka Connect
Move data between external systems and Kafka using source and sink connectors, workers and tasks.
Apache Flink
Build streaming transformations, aggregations and windowed computations with event-time concepts.
Confluent Cloud
Explore managed Kafka, access configuration, hosted schema services, connectors and available Flink capabilities.
Production-minded skills
Practice retries, idempotency, error handling, dead-letter topics, monitoring and troubleshooting.
15-day syllabus
Step-by-step learning plan
Select any day to view topics and the practical lab. Each session is planned for approximately one hour.
DAY 01Kafka FundamentalsArchitecture and event streaming
- Event streaming and real-world use cases
- Kafka brokers, topics, producers and consumers
- Kafka compared with traditional message queues
Lab: Trace an e-commerce order event from producer to consumer.
DAY 02Topics, Partitions & OffsetsOrdering, retention and replication
- Partitioning, keys and ordering
- Offsets, retention and replication
- Consumer groups and rebalancing
Lab: Create a topic, publish keyed messages and inspect offsets.
DAY 03Producers & ConsumersAPIs and message processing
- Serialization and deserialization
- Consumer polling and offset commits
- Delivery guarantees and retries
Lab: Write a Java producer and consumer.
DAY 04Reliability & ScalingFailures and delivery semantics
- At-most-once, at-least-once and exactly-once semantics
- Idempotency, duplicates and retries
- Dead-letter topics and consumer lag
Lab: Simulate consumer failure and implement error handling.
DAY 05Kafka with Spring BootBuild an application
KafkaTemplateand@KafkaListener- Producer and consumer configuration
- Application-level exception handling
Lab: Build a Spring Boot order service and consumer.
DAY 06Schema Registry — Part 1Schemas and serialization
- Why schemas are needed
- JSON vs Avro vs Protobuf
- Subjects, versions and schema IDs
Lab: Register an order schema and publish an Avro event.
DAY 07Schema EvolutionCompatibility and versioning
- Backward, forward and full compatibility
- Optional fields and schema changes
- Handling incompatible changes
Lab: Add a field and test compatibility with an earlier schema.
DAY 08Kafka Connect — Part 1Connectors, workers and tasks
- Source vs sink connectors
- Standalone vs distributed workers
- Connector and task configuration
Lab: Ingest data using an available file or database connector.
DAY 09Kafka Connect — Part 2Integration and troubleshooting
- Connector REST API and lifecycle
- Sink integration and transformations
- Retries, errors and dead-letter queues
Lab: Send events to a sink and troubleshoot a failed task.
DAY 10Flink FundamentalsStream processing concepts
- Flink vs batch processing
- Jobs, operators and data streams
- Kafka as a source and sink
Lab: Read Kafka events and transform them in Flink.
DAY 11Flink State & WindowsEvent time and aggregation
- Event time vs processing time
- Windows, watermarks and late events
- Keyed state and checkpoints
Lab: Calculate orders per minute and total order value.
DAY 12Flink SQL & RecoveryStreaming queries and operations
- Flink SQL and table concepts
- Continuous queries and output topics
- Job recovery and checkpointing
Lab: Run a streaming aggregation and write results to Kafka.
DAY 13Confluent Cloud — Part 1Managed Kafka and security
- Managed vs self-managed Kafka
- Environments, clusters and topics
- API keys, service accounts and access control
Lab: Connect a Java producer and consumer to a provisioned cloud cluster.
DAY 14Confluent Cloud — Part 2Schemas, connectors and Flink
- Managed Schema Registry
- Connectors and data integration
- Flink capabilities where enabled
- Monitoring, usage and security basics
Lab: Configure a schema-aware client and run an available cloud data service.
DAY 15Capstone & AssessmentEnd-to-end project presentation
- Integrate producers, consumers and schemas
- Connect a data source or sink
- Process streaming data with Flink
- Debug and present the architecture
Lab: Demonstrate the full e-commerce event streaming workflow.
Capstone project
Build an e-commerce event streaming pipeline
Use one practical scenario throughout the course to connect each concept to a real application.
Example event flow
Order ServicePublishes OrderCreated events
Kafka topic: ordersStores partitioned event records
Consumers + FlinkProcess payments, stock and analytics
What learners will implement
- Publish order events with an order ID, customer ID, amount and status.
- Use record keys to preserve ordering for events belonging to the same order.
- Consume events through independent consumer groups.
- Register schemas and test a compatible schema change.
- Integrate a source or sink through Kafka Connect.
- Aggregate streaming data with Flink.
- Connect an application to Confluent Cloud where access is available.
- Handle retries, duplicate processing and failed messages.
Learning outcomes
Skills learners can demonstrate
Build Kafka applicationsCreate and troubleshoot producers and consumers.
Manage schema changesUse Schema Registry and compatibility rules.
Integrate data systemsConfigure source and sink connectors.
Process event streamsBuild basic Flink transformations and aggregations.
Work with managed KafkaConnect securely to Confluent Cloud.
Debug common failuresUnderstand retries, offsets, lag and recovery.
Frequently asked questions
Before you get started
Who should take this course?
Interns, fresh graduates and developers who want a practical introduction to event-driven applications and real-time data streaming.
What should I know before starting?
Basic programming, Java fundamentals, JSON and command-line usage are recommended. The labs use Java and Spring Boot examples.
Is the course theory-only?
No. The course includes daily hands-on exercises and a capstone project. The exact tools available for each lab depend on the training environment.
Do I need a Confluent Cloud account?
Cloud labs require an appropriately configured environment and permissions. Availability and costs should be confirmed before the training begins.
How can I get the schedule and fees?
Contact Cloudhill to discuss batch dates, delivery format, fees and any custom requirements.
Ready to learn real-time data streaming?
Contact Cloudhill for course details, upcoming batches and training enquiries.
Course duration and topics are a proposed training plan. Specific cloud services, connector availability and lab requirements depend on the selected environment and service plan.