org.apache.spark.sql.connector.read

Interface SupportsPushDownTopN

All Superinterfaces:

ScanBuilder

@Evolving
public interface SupportsPushDownTopN
extends ScanBuilder

A mix-in interface for ScanBuilder. Data sources can implement this interface to push down top N(query with ORDER BY ... LIMIT n). Please note that the combination of top N with other operations such as AGGREGATE, GROUP BY, CLUSTER BY, DISTRIBUTE BY, etc. is NOT pushed down.

Since:

3.3.0

Method Summary

Modifier and Type	Method and Description
default boolean	isPartiallyPushed() Whether the top N is partially pushed or not.
boolean	pushTopN(SortOrder[] orders, int limit) Pushes down top N to the data source.

Methods inherited from interface org.apache.spark.sql.connector.read.ScanBuilder

build

Method Detail

pushTopN

boolean pushTopN(SortOrder[] orders,
                 int limit)

Pushes down top N to the data source.

isPartiallyPushed

default boolean isPartiallyPushed()

Whether the top N is partially pushed or not. If it returns true, then Spark will do top N again. This method will only be called when pushTopN(org.apache.spark.sql.connector.expressions.SortOrder[], int) returns true.