org.apache.spark.sql

Class SparkSession.Builder

java.lang.Object

org.apache.spark.sql.SparkSession.Builder

Enclosing class:

SparkSession

public static class SparkSession.Builder
extends java.lang.Object

Builder for SparkSession.

Constructor Summary

Constructors

Constructor and Description
SparkSession.Builder()

Method Summary

Methods

Modifier and Type	Method and Description
SparkSession.Builder	appName(java.lang.String name) Sets a name for the application, which will be shown in the Spark web UI.
SparkSession.Builder	config(SparkConf conf) Sets a list of config options based on the given SparkConf.
SparkSession.Builder	config(java.lang.String key, boolean value) Sets a config option.
SparkSession.Builder	config(java.lang.String key, double value) Sets a config option.
SparkSession.Builder	config(java.lang.String key, long value) Sets a config option.
SparkSession.Builder	config(java.lang.String key, java.lang.String value) Sets a config option.
SparkSession.Builder	enableHiveSupport() Enables Hive support, including connectivity to a persistent Hive metastore, support for Hive serdes, and Hive user-defined functions.
SparkSession	getOrCreate() Gets an existing SparkSession or, if there is no existing one, creates a new one based on the options set in this builder.
SparkSession.Builder	master(java.lang.String master) Sets the Spark master URL to connect to, such as "local" to run locally, "local[4]" to run locally with 4 cores, or "spark://master:7077" to run on a Spark standalone cluster.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkSession.Builder

public SparkSession.Builder()

Method Detail

appName

public SparkSession.Builder appName(java.lang.String name)

Sets a name for the application, which will be shown in the Spark web UI.

Parameters:

name - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

config

public SparkSession.Builder config(java.lang.String key,
                          java.lang.String value)

Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration.

Parameters:

key - (undocumented)

value - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

config

public SparkSession.Builder config(java.lang.String key,
                          long value)

Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration.

Parameters:

key - (undocumented)

value - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

config

public SparkSession.Builder config(java.lang.String key,
                          double value)

Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration.

Parameters:

key - (undocumented)

value - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

config

public SparkSession.Builder config(java.lang.String key,
                          boolean value)

Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration.

Parameters:

key - (undocumented)

value - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

config

public SparkSession.Builder config(SparkConf conf)

Sets a list of config options based on the given SparkConf.

Parameters:

conf - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

master

public SparkSession.Builder master(java.lang.String master)

Sets the Spark master URL to connect to, such as "local" to run locally, "local[4]" to run locally with 4 cores, or "spark://master:7077" to run on a Spark standalone cluster.

Parameters:

master - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

enableHiveSupport

public SparkSession.Builder enableHiveSupport()

Enables Hive support, including connectivity to a persistent Hive metastore, support for Hive serdes, and Hive user-defined functions.

Returns:

(undocumented)

Since:

2.0.0

getOrCreate

public SparkSession getOrCreate()

Gets an existing SparkSession or, if there is no existing one, creates a new one based on the options set in this builder.

Returns:

(undocumented)

Since:

2.0.0