Interface StateRestoreListener


@Public public interface StateRestoreListener
Class for listening to various states of the restoration process of a StateStore.

When calling KafkaStreams.setGlobalStateRestoreListener(StateRestoreListener) the passed instance is expected to be stateless since the StateRestoreListener is shared across all state updater threads, of which there is one per stream thread (num.stream.threads).

Users desiring stateful operations will need to provide synchronization internally in the StateRestoreListener implementation.

Note that this listener is only registered at the per-client level and users can use the storeName parameter to define specific monitoring for different StateStores.

StateRestoreListener must not be confused with the StateRestoreCallback interface. The StateRestoreListener is a callback to observe restoration progress. In contrast, the StateRestoreCallback interface is used by custom state store implementations to provide the actual state restoration functionality.

Also note that the update process of standby tasks is not monitored via this interface, since a standby task does not actually restore state, but keeps updating its state from the changelogs written by the active task which does not ever finish. The same holds for warm-up tasks, which the runtime runs as standby tasks. Both are monitored via StandbyUpdateListener instead.

Incremental updates are exposed so users can estimate how much progress has been made.

See Also:
  • Method Summary

    Modifier and Type
    Method
    Description
    void
    onBatchRestored(TopicPartition topicPartition, String storeName, long batchEndOffset, long numRestored)
    Method called after restoring a batch of records.
    void
    onRestoreEnd(TopicPartition topicPartition, String storeName, long totalRestored)
    Method called when restoring the StateStore is complete.
    void
    onRestoreStart(TopicPartition topicPartition, String storeName, long startingOffset, long endingOffset)
    Method called at the very beginning of StateStore restoration.
    default void
    onRestoreSuspended(TopicPartition topicPartition, String storeName, long totalRestored)
    Method called when restoring the StateStore is suspended before it finished, because the task stopped being an active task on this host: it was assigned elsewhere, demoted to a standby task, or this host is shutting down (the concrete reason is not reported).
  • Method Details

    • onRestoreStart

      void onRestoreStart(TopicPartition topicPartition, String storeName, long startingOffset, long endingOffset)
      Method called at the very beginning of StateStore restoration.

      Called for every active task that begins restoring, including one that has nothing to restore because its local state is already up to date -- a task whose state was just built up as a standby or warm-up task, for instance. In that case startingOffset equals endingOffset and onRestoreEnd(TopicPartition, String, long) follows with a total of zero records restored, so an empty restoration is told apart by the offsets and the count rather than by the callbacks being skipped.

      Parameters:
      topicPartition - the TopicPartition containing the values to restore
      storeName - the name of the store undergoing restoration
      startingOffset - the starting offset of the entire restoration process for this TopicPartition
      endingOffset - the exclusive ending offset of the entire restoration process for this TopicPartition
    • onBatchRestored

      void onBatchRestored(TopicPartition topicPartition, String storeName, long batchEndOffset, long numRestored)
      Method called after restoring a batch of records. In this case the maximum size of the batch is whatever the value of max.poll.records is set to.

      This method is called after restoring each batch and it is advised to keep processing to a minimum. Any heavy processing will block the state updater thread and hold up recovering the next batch, hence slowing down the restore process as a whole. If you need to do any extended processing or connecting to an external service consider doing so asynchronously.

      Parameters:
      topicPartition - the TopicPartition containing the values to restore
      storeName - the name of the store undergoing restoration
      batchEndOffset - the inclusive ending offset for the current restored batch for this TopicPartition
      numRestored - the total number of records restored in this batch for this TopicPartition
    • onRestoreEnd

      void onRestoreEnd(TopicPartition topicPartition, String storeName, long totalRestored)
      Method called when restoring the StateStore is complete.
      Parameters:
      topicPartition - the TopicPartition containing the values to restore
      storeName - the name of the store just restored
      totalRestored - the total number of records restored for this TopicPartition
    • onRestoreSuspended

      default void onRestoreSuspended(TopicPartition topicPartition, String storeName, long totalRestored)
      Method called when restoring the StateStore is suspended before it finished, because the task stopped being an active task on this host: it was assigned elsewhere, demoted to a standby task, or this host is shutting down (the concrete reason is not reported). If the task becomes an active task on this host again -- assigned back to it, or promoted from the standby task it was demoted to -- another onRestoreStart(TopicPartition, String, long, long) would be called.

      A task whose restoration had already finished, and was reported through onRestoreEnd(TopicPartition, String, long), is not reported here when it is closed later on. In particular, an active task that had finished restoring and transitioned into the RUNNING state and is then demoted to a standby task is not reported here at all; it surfaces only as a StandbyUpdateListener.onUpdateStart(TopicPartition, String, long) for the standby task that replaces it.

      Parameters:
      topicPartition - the TopicPartition containing the values to restore
      storeName - the name of the store just restored
      totalRestored - the total number of records restored for this TopicPartition before being suspended