org.apache.spark.sql.parquet

Class ParquetTableScan

Object

org.apache.spark.sql.catalyst.trees.TreeNode<PlanType>

org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

org.apache.spark.sql.execution.SparkPlan

org.apache.spark.sql.parquet.ParquetTableScan

All Implemented Interfaces:

java.io.Serializable, Logging, org.apache.spark.sql.catalyst.trees.LeafNode<SparkPlan>, scala.Equals, scala.Product

public class ParquetTableScan
extends SparkPlan
implements LeafNode, scala.Product, scala.Serializable

:: DeveloperApi :: Parquet table scan operator. Imports the file that backs the given ParquetRelation as a RDD[Row].

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ParquetTableScan(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes, ParquetRelation relation, scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred)

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute>	attributes()
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression>	columnPruningPred()
RDD<org.apache.spark.sql.catalyst.expressions.Row>	execute() Runs this query returning the result as an RDD.
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute>	output()
ParquetTableScan	pruneColumns(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> prunedAttributes) Applies a (candidate) projection.
ParquetRelation	relation()
scala.Tuple2<Object,Object>[]	requestedPartitionOrdinals()

Methods inherited from class org.apache.spark.sql.execution.SparkPlan

codegenEnabled, executeCollect, makeCopy, outputPartitioning, requiredChildDistribution

Methods inherited from class org.apache.spark.sql.catalyst.plans.QueryPlan

expressions, inputSet, missingInput, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionDown$1, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionUp$1, outputSet, printSchema, references, schema, schemaString, simpleString, statePrefix, transformAllExpressions, transformExpressions, transformExpressionsDown, transformExpressionsUp

Methods inherited from class org.apache.spark.sql.catalyst.trees.TreeNode

apply, argString, asCode, children, collect, fastEquals, flatMap, foreach, generateTreeString, getNodeNumbered, map, mapChildren, nodeName, numberedTreeString, otherCopyArgs, stringArgs, toString, transform, transformChildrenDown, transformChildrenUp, transformDown, transformUp, treeString, withNewChildren

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.catalyst.trees.LeafNode

children

Methods inherited from interface scala.Product

productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

ParquetTableScan

public ParquetTableScan(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes,
                ParquetRelation relation,
                scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred)

Method Detail

attributes

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes()

relation

public ParquetRelation relation()

columnPruningPred

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred()

output

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output()

Specified by:

output in class org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

requestedPartitionOrdinals

public scala.Tuple2<Object,Object>[] requestedPartitionOrdinals()

execute

public RDD<org.apache.spark.sql.catalyst.expressions.Row> execute()

Description copied from class: SparkPlan

Runs this query returning the result as an RDD.

Specified by:

execute in class SparkPlan

pruneColumns

public ParquetTableScan pruneColumns(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> prunedAttributes)

Applies a (candidate) projection.

Parameters:

prunedAttributes - The list of attributes to be used in the projection.

Returns:

Pruned TableScan.