Skip to content

Advanced Examples

Explore KSTD designs of different types of Kafka Streams applications, ranging from pure DSL topologies, to low level implementation of the Processor API and state stores.

Examples

Word Count

The WordCount Demo design is a relatively simple example and a good starting point.

To find more details on this example, including a step-by-step guide to recreate the diagram by yourself, please start with the tutorial.

KSTD 'WordCount Demo' tutorial - final result

Ecommerce Product View

This Kafka Streams topology creates a full-context view for a product of an Ecommerce application from 5 data streams. The source topics have different no. of partitions, and also different keys and scope.

Through transformations, joins, and aggregations, all data streams are combined to create a view of a product that may come in multiple variants (e.g. colors), each with its own price, stock, and availability. Variants form a ‘product group’ that have certain product details (description) and various attributes in common.

'Kafka Streams Topology Design' diagram of an eCommerce example app, producing a full-context 'product view', both as avro and json

Events

  • ProductDescription Event: Provides basic product information such as name, description, and category.
  • ProductAttribute Event: Represents attributes of a product such as material, color options, or features.
  • ProductVariant Event: Defines product variants such as different sizes or colors for the same product.
  • ProductVariantPrice Event: Provides pricing information for a specific product variant, including discounts.
  • ProductVariantStock Event: Provides stock and availability information for a product variant.
  • ProductFullContext Event: This event holds all information about a product, including its description, attributes, variants, pricing, stock, and availability. It is designed to offer a full-context view of a product in the eCommerce system, which can be used for enriching product displays, recommendations, or inventory management.

User Flight Booking - Status Update Notifications

A travel company offers booking flights through their website. To notify users about any changes or updates to their flight bookings in real-time, a stream processing application is implemented.

Flight status updates, such as ‘check-in started’, or ‘flight cancelled’, have to be communicated to all relevant users in their configured communication channels (email, app, text message). The notification text should include event details, as well as information on the departure, layover, and destination airports in both English and in their preferred language.

Data streams are available as three Kafka topics. User notifications are triggered via a ‘Notification Service’ REST API.

'Kafka Streams Topology Design' diagram for the example app 'User Flight Booking - Status Update Notifications'

Events

  • AirportInfoI18n Event: transports information about airports globally in different languages. The schema includes mandatory information in English, with the option to include translations in other languages.
  • FlightStatusUpdate Event: focuses on the flight as a whole rather than just the departure airport. It includes information about the current status of the flight, any changes to the flight schedule, and delays.
  • UserFlightBooking Event: holds information about a traveler’s booked flight, including reservation details, contact information, and preferred language for notifications.
  • UserFlightBookingNotification Event: joins and combines information from UserFlightBooking, Flight, FlightStatusUpdate, and AirportInfoI18n into an enriched output event, which is emitted on all flight status updates

Disclaimer ❗

The examples presented on this page demonstrate advanced Kafka Streams topologies. While they address real-world challenges, please note that these scenarios are fictional, designed to illustrate practical applications of the KSTD standardnot to provide a comprehensive tutorial on Kafka Streams itself.

The designs were created with the author’s best efforts to be both realistic and implementable within the outlined requirements. However, they may not represent the optimal or most efficient solutions.