State Stores

Components

State Store

State Store is a local storage used in Kafka Streams to store and retrieve stateful information, allowing stateful operations like aggregations and joins.

State Store Types

In-Memory Store is a state store type that holds data in the application's memory, providing fast access but limited by memory size and volatility.

RocksDB Store is a persistent state store backed by RocksDB, enabling efficient storage and retrieval of large datasets on disk.

LRU Cache Store is a state store type that uses an in-memory cache with a Least Recently Used (LRU) eviction policy, providing fast access with limited memory footprint.

State Store Variants

Global Store is a variant of state store that is replicated to all instances of the Kafka Streams application, allowing global access to state data across partitions.

Versioned Store is a state store variant that maintains multiple versions of values over time, enabling access to historical state values for each key.

Read-Only Store is a variant of state store that allows read access but prevents modifications, typically used in queryable state scenarios. (ref KIP-813)

State Store Attribute - Logged

Logged State Store is a state store where changes are automatically logged to a Kafka changelog topic, enabling fault tolerance and recovery.

Non-Logged State Store is a state store without Kafka-backed logging.

State Store Attribute - Cached

Cached State Store is a state store with an in-memory write buffer. Updates to the state store are buffered and only flushed to disk (when using RocksDB) and changelog topic (when logging enabled) on buffer overflow or when offsets are committed.

Non-Cached State Store is a state store that does not buffer, essentially flushes and forwards immediatly after processing.