org.apache.spark.sql.execution

Class PythonUDF

Object

org.apache.spark.sql.catalyst.trees.TreeNode<org.apache.spark.sql.catalyst.expressions.Expression>

org.apache.spark.sql.catalyst.expressions.Expression

org.apache.spark.sql.execution.PythonUDF

All Implemented Interfaces:

java.io.Serializable, Logging, scala.Equals, scala.Product

public class PythonUDF
extends org.apache.spark.sql.catalyst.expressions.Expression
implements Logging, scala.Product, scala.Serializable

A serialized version of a Python lambda function. Suitable for use in a PythonRDD.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description

PythonUDF(String name, byte[] command, java.util.Map<String,String> envVars, java.util.List<String> pythonIncludes, String pythonExec, java.util.List<Broadcast<org.apache.spark.api.python.PythonBroadcast>> broadcastVars, Accumulator<java.util.List<byte[]>> accumulator, org.apache.spark.sql.catalyst.types.DataType dataType, scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> children)

Method Summary

Methods

Modifier and Type	Method and Description
Accumulator<java.util.List<byte[]>>	accumulator()
java.util.List<Broadcast<org.apache.spark.api.python.PythonBroadcast>>	broadcastVars()
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression>	children()
byte[]	command()
org.apache.spark.sql.catalyst.types.DataType	dataType()
java.util.Map<String,String>	envVars()
scala.runtime.Nothing$	eval(org.apache.spark.sql.catalyst.expressions.Row input)
String	name()
boolean	nullable()
String	pythonExec()
java.util.List<String>	pythonIncludes()
String	toString()

Methods inherited from class org.apache.spark.sql.catalyst.expressions.Expression

c2, childrenResolved, eval$default$1, f1, f2, foldable, i1, i2, n1, n2, references, resolved

Methods inherited from class org.apache.spark.sql.catalyst.trees.TreeNode

apply, argString, asCode, collect, fastEquals, flatMap, foreach, generateTreeString, getNodeNumbered, makeCopy, map, mapChildren, nodeName, numberedTreeString, otherCopyArgs, simpleString, stringArgs, transform, transformChildrenDown, transformChildrenUp, transformDown, transformUp, treeString, withNewChildren

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Methods inherited from interface scala.Product

productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Constructor Detail

PythonUDF

public PythonUDF(String name,
         byte[] command,
         java.util.Map<String,String> envVars,
         java.util.List<String> pythonIncludes,
         String pythonExec,
         java.util.List<Broadcast<org.apache.spark.api.python.PythonBroadcast>> broadcastVars,
         Accumulator<java.util.List<byte[]>> accumulator,
         org.apache.spark.sql.catalyst.types.DataType dataType,
         scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> children)

Method Detail

name

public String name()

command

public byte[] command()

envVars

public java.util.Map<String,String> envVars()

pythonIncludes

public java.util.List<String> pythonIncludes()

pythonExec

public String pythonExec()

broadcastVars

public java.util.List<Broadcast<org.apache.spark.api.python.PythonBroadcast>> broadcastVars()

accumulator

public Accumulator<java.util.List<byte[]>> accumulator()

dataType

public org.apache.spark.sql.catalyst.types.DataType dataType()

Specified by:

dataType in class org.apache.spark.sql.catalyst.expressions.Expression

children

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> children()

Specified by:

children in class org.apache.spark.sql.catalyst.trees.TreeNode<org.apache.spark.sql.catalyst.expressions.Expression>

toString

public String toString()

Overrides:

toString in class org.apache.spark.sql.catalyst.trees.TreeNode<org.apache.spark.sql.catalyst.expressions.Expression>

nullable

public boolean nullable()

Specified by:

nullable in class org.apache.spark.sql.catalyst.expressions.Expression

eval

public scala.runtime.Nothing$ eval(org.apache.spark.sql.catalyst.expressions.Row input)

Specified by:

eval in class org.apache.spark.sql.catalyst.expressions.Expression