(Ebook) Streaming Data Pipelines with Kafka (MEAP) by Stefan Sprenger ISBN 9781633437012, 1633437019

(Ebook) Streaming Data Pipelines with Kafka (MEAP) by Stefan Sprenger ISBN 9781633437012, 1633437019

Streaming Data Pipelines with Apache Kafka teaches you to build the kind of rapid, reliable data pipelines that can deliver real-time insights from your data. You’ll follow along with an extended case study as Excellent Toys Corporation's data team migrates from batch processing to their very first streaming pipelines. Dive into custom connector development, extracting real-time changes from an HTTP-based Analytics API, and delve into event-driven, real-time processing with Kafka Streams. With guidance on packaging, deploying, and error handling, you'll soon be equipped to build and deploy streaming data pipelines in production environments. Deliver real-time insights into your data with a rapid, reliable streaming data pipeline. Streaming data pipelines let you integrate data from multiple systems in real time, with instantaneously updating and processing from data source to data sink. In Streaming Data Pipelines with Kafka you’ll build the kind of streaming pipelines that hold up modern data infrastructure, all with the industry-standard Apache Kafka platform. Inside this practical guide, you’ll learn how to:• Serve real-time data to business departments of your organization• Understand streaming data pipeline concepts such as change data capture• Troubleshoot common challenges when building and deploying streaming data pipelines• Setup open-source connectors with Kafka Connect and develop custom connectors yourself• Implement stateless and stateful data processing with Kafka Streams• Tune pipeline performance for low-latency and high-throughput requirements• Scale pipelines both manually and automatically to cope with performance requirements• Debug and monitor streaming data pipelines in production• Decide when to use streaming data pipelines over batch pipelines