org.apache.spark.sql.sources.v2.reader.streaming

Interface ContinuousDataReader<T>

All Superinterfaces:

AutoCloseable, java.io.Closeable, DataReader<T>

@InterfaceStability.Evolving
public interface ContinuousDataReader<T>
extends DataReader<T>

A variation on DataReader for use with streaming in continuous processing mode.

Method Summary

Modifier and Type	Method and Description
PartitionOffset	getOffset() Get the offset of the current record, or the start offset if no records have been read.

Methods inherited from interface org.apache.spark.sql.sources.v2.reader.DataReader

get, next

Methods inherited from interface java.io.Closeable

close

Method Detail

getOffset

PartitionOffset getOffset()

Get the offset of the current record, or the start offset if no records have been read. The execution engine will call this method along with get() to keep track of the current offset. When an epoch ends, the offset of the previous record in each partition will be saved as a restart checkpoint.