Interface SupportsReportOrdering
- All Superinterfaces:
Scan
A mix in interface for
Scan. Data sources can implement this interface to
report the order of data in each partition to Spark.
Global order is part of the partitioning, see SupportsReportPartitioning.
Spark uses ordering information to exploit existing order to avoid sorting required by subsequent operations.
- Since:
- 3.4.0
-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.spark.sql.connector.read.Scan
Scan.ColumnarSupportMode -
Method Summary
Modifier and TypeMethodDescriptionReturns the order in each partition of this data source scan.Methods inherited from interface org.apache.spark.sql.connector.read.Scan
columnarSupportMode, description, readSchema, reportDriverMetrics, supportedCustomMetrics, toBatch, toContinuousStream, toMicroBatchStream
-
Method Details
-
outputOrdering
SortOrder[] outputOrdering()Returns the order in each partition of this data source scan.Spark currently does not rely on a sort order over a transform such as
days(ts)to avoid a sort, whether or not it is a partition key. Nor does it rely on the sort orders after it. The exception is a sort order on a partition key that is not a transform, when Spark uses the reportedKeyGroupedPartitioning.
-