org.apache.spark.sql.sources.v2.reader.streaming

Interface ContinuousInputPartitionReader<T>

All Superinterfaces:

AutoCloseable, java.io.Closeable, InputPartitionReader<T>

@InterfaceStability.Evolving
public interface ContinuousInputPartitionReader<T>
extends InputPartitionReader<T>

A variation on InputPartitionReader for use with streaming in continuous processing mode.

Method Summary

Modifier and Type	Method and Description
PartitionOffset	getOffset() Get the offset of the current record, or the start offset if no records have been read.

Methods inherited from interface org.apache.spark.sql.sources.v2.reader.InputPartitionReader

get, next

Methods inherited from interface java.io.Closeable

close

Method Detail

getOffset

PartitionOffset getOffset()

Get the offset of the current record, or the start offset if no records have been read. The execution engine will call this method along with get() to keep track of the current offset. When an epoch ends, the offset of the previous record in each partition will be saved as a restart checkpoint.