Interface StandbyUpdateListener
When calling KafkaStreams.setStandbyUpdateListener(StandbyUpdateListener)
the passed instance is expected to be stateless since the StandbyUpdateListener is shared across all state
updater threads, of which there is one per stream thread (num.stream.threads).
Users desiring stateful operations will need to provide synchronization internally in
the StandbyUpdateListener implementation.
Note that this listener is only registered at the per-client level and users can use the storeName
parameter to define specific monitoring for different StateStores.
Warm-up tasks are reported through this interface as well as standby tasks.
A warm-up task is one that the group hands out so that this client can build a task's state up before taking the task
over, and the runtime runs it as a standby task, so the two are indistinguishable to this listener.
Relabelling a task between the two roles does not restart the task, and does not trigger a callback; a task might be
converted from a warm-up task to a standby task between two calls of onBatchLoaded(TopicPartition, String, TaskId, long, long, long) transparently.
Also note that, unlike the restoration of an active task, the update of a standby task never finishes, since it
keeps applying the changelog records written by the active task.
There is therefore no callback corresponding to
StateRestoreListener.onRestoreEnd(TopicPartition, String, long); a standby task is reported only once it
stops being updated, through onUpdateSuspended(TopicPartition, String, long, long, SuspendReason).
Even if warm-up tasks are expected to "complete" restoration, the same pattern as for standby tasks applies.
Once a warm-up task is considered hot (based on acceptable.recovery.lag), its restoration will be
suspended when it is promoted to an active task, and the active task will complete the restoration (if
registered, the corresponding StateRestoreListener callback is invoked on the active task for this
case).
Incremental updates are exposed so users can estimate how much progress has been made.
- See Also:
-
Nested Class Summary
Nested ClassesModifier and TypeInterfaceDescriptionstatic enumThe reason why a standby or warm-up task stopped being updated. -
Method Summary
Modifier and TypeMethodDescriptionvoidonBatchLoaded(TopicPartition topicPartition, String storeName, TaskId taskId, long batchEndOffset, long batchSize, long currentEndOffset) Method called after loading a batch of records.voidonUpdateStart(TopicPartition topicPartition, String storeName, long startingOffset) Method called at the very beginning ofStateStoreupdating.voidonUpdateSuspended(TopicPartition topicPartition, String storeName, long storeOffset, long currentEndOffset, StandbyUpdateListener.SuspendReason reason) Method called when the corresponding standby or warm-up task stops being updated, for the provided reason.
-
Method Details
-
onUpdateStart
Method called at the very beginning ofStateStoreupdating.- Parameters:
topicPartition- the changelogTopicPartitionfor this taskstoreName- the name of the store being loadedstartingOffset- the offset from which the task begins consuming from the changelog
-
onBatchLoaded
void onBatchLoaded(TopicPartition topicPartition, String storeName, TaskId taskId, long batchEndOffset, long batchSize, long currentEndOffset) Method called after loading a batch of records. In this case the maximum size of the batch is whatever the value ofmax.poll.recordsis set to.This method is called after loading each batch and it is advised to keep processing to a minimum. Any heavy processing will block the state updater thread and slow down the rate of standby task loading. Therefore, if you need to do any extended processing or connect to an external service, consider doing so asynchronously.
- Parameters:
topicPartition- the changelogTopicPartitionfor this taskstoreName- the name of the store being loadedtaskId- theTaskIdof the task that owns the store being loadedbatchEndOffset- the changelog end offset (inclusive) of the batch that was just loadedbatchSize- the total number of records in the batch that was just loadedcurrentEndOffset- the current end offset of the changelog topic partition
-
onUpdateSuspended
void onUpdateSuspended(TopicPartition topicPartition, String storeName, long storeOffset, long currentEndOffset, StandbyUpdateListener.SuspendReason reason) Method called when the corresponding standby or warm-up task stops being updated, for the provided reason.If the reason is
StandbyUpdateListener.SuspendReason.MIGRATED, this callback will be invoked after this state store (and the task itself) are closed, in which case the data will be cleaned up afterstate.cleanup.delay.ms. If the task wasStandbyUpdateListener.SuspendReason.PROMOTEDto an active task, the state store will not be closed, and the callback will be invoked after unregistering it as a standby task but before re-registering it as an active task and beginning restoration. In other words, this will always be called before the correspondingStateRestoreListener.onRestoreStart(TopicPartition, String, long, long)call is made.- Parameters:
topicPartition- the changelogTopicPartitionfor this taskstoreName- the name of the store being loadedstoreOffset- the offset of the last changelog record that was read and put into the store at the time of suspensioncurrentEndOffset- the current end offset of the changelog topic partitionreason- the reason why the task stopped being updated
-