org.apache.spark.sql.sources.v2.reader.streaming

Class Offset

Object

org.apache.spark.sql.execution.streaming.Offset

org.apache.spark.sql.sources.v2.reader.streaming.Offset

@InterfaceStability.Evolving
public abstract class Offset
extends org.apache.spark.sql.execution.streaming.Offset

An abstract representation of progress through a MicroBatchReader or ContinuousReader. During execution, offsets provided by the data source implementation will be logged and used as restart checkpoints. Each source should provide an offset implementation which the source can use to reconstruct a position in the stream up to which data has been seen/processed. Note: This class currently extends Offset to maintain compatibility with DataSource V1 APIs. This extension will be removed once we get rid of V1 completely.

Constructor Summary

Constructors

Constructor and Description
Offset()

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object obj) Equality based on JSON string representation.
int	hashCode()
abstract String	json() A JSON-serialized representation of an Offset that is used for saving offsets to the offset log.
String	toString()

Methods inherited from class Object

getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

Offset

public Offset()

Method Detail

json

public abstract String json()

A JSON-serialized representation of an Offset that is used for saving offsets to the offset log. Note: We assume that equivalent/equal offsets serialize to identical JSON strings.

Specified by:

json in class org.apache.spark.sql.execution.streaming.Offset

Returns:

JSON string encoding

equals

public boolean equals(Object obj)

Equality based on JSON string representation. We leverage the JSON representation for normalization between the Offset's in deserialized and serialized representations.

Overrides:

equals in class org.apache.spark.sql.execution.streaming.Offset

hashCode

public int hashCode()

Overrides:

hashCode in class org.apache.spark.sql.execution.streaming.Offset

toString

public String toString()

Overrides:

toString in class org.apache.spark.sql.execution.streaming.Offset