org.apache.spark.sql.catalog

Class Catalog

java.lang.Object

org.apache.spark.sql.catalog.Catalog

Direct Known Subclasses:

CatalogImpl

public abstract class Catalog
extends java.lang.Object

Catalog interface for Spark. To access this, use SparkSession.catalog.

Constructor Summary

Constructors

Constructor and Description
Catalog()

Method Summary

Methods

Modifier and Type	Method and Description
abstract void	cacheTable(java.lang.String tableName) Caches the specified table in-memory.
abstract void	clearCache() Removes all cached tables from the in-memory cache.
abstract Dataset<Row>	createExternalTable(java.lang.String tableName, java.lang.String path) :: Experimental :: Creates an external table from the given path and returns the corresponding DataFrame.
abstract Dataset<Row>	createExternalTable(java.lang.String tableName, java.lang.String source, java.util.Map<java.lang.String,java.lang.String> options) :: Experimental :: Creates an external table from the given path based on a data source and a set of options.
abstract Dataset<Row>	createExternalTable(java.lang.String tableName, java.lang.String source, scala.collection.immutable.Map<java.lang.String,java.lang.String> options) :: Experimental :: (Scala-specific) Creates an external table from the given path based on a data source and a set of options.
abstract Dataset<Row>	createExternalTable(java.lang.String tableName, java.lang.String path, java.lang.String source) :: Experimental :: Creates an external table from the given path based on a data source and returns the corresponding DataFrame.
abstract Dataset<Row>	createExternalTable(java.lang.String tableName, java.lang.String source, StructType schema, java.util.Map<java.lang.String,java.lang.String> options) :: Experimental :: Create an external table from the given path based on a data source, a schema and a set of options.
abstract Dataset<Row>	createExternalTable(java.lang.String tableName, java.lang.String source, StructType schema, scala.collection.immutable.Map<java.lang.String,java.lang.String> options) :: Experimental :: (Scala-specific) Create an external table from the given path based on a data source, a schema and a set of options.
abstract java.lang.String	currentDatabase() Returns the current default database in this session.
abstract void	dropTempView(java.lang.String viewName) Drops the temporary view with the given view name in the catalog.
abstract boolean	isCached(java.lang.String tableName) Returns true if the table is currently cached in-memory.
abstract Dataset<Column>	listColumns(java.lang.String tableName) Returns a list of columns for the given table in the current database.
abstract Dataset<Column>	listColumns(java.lang.String dbName, java.lang.String tableName) Returns a list of columns for the given table in the specified database.
abstract Dataset<Database>	listDatabases() Returns a list of databases available across all sessions.
abstract Dataset<Function>	listFunctions() Returns a list of functions registered in the current database.
abstract Dataset<Function>	listFunctions(java.lang.String dbName) Returns a list of functions registered in the specified database.
abstract Dataset<Table>	listTables() Returns a list of tables in the current database.
abstract Dataset<Table>	listTables(java.lang.String dbName) Returns a list of tables in the specified database.
abstract void	setCurrentDatabase(java.lang.String dbName) Sets the current default database in this session.
abstract void	uncacheTable(java.lang.String tableName) Removes the specified table from the in-memory cache.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Catalog

public Catalog()

Method Detail

currentDatabase

public abstract java.lang.String currentDatabase()

Returns the current default database in this session.

Returns:

(undocumented)

Since:

2.0.0

setCurrentDatabase

public abstract void setCurrentDatabase(java.lang.String dbName)

Sets the current default database in this session.

Parameters:

dbName - (undocumented)

Since:

2.0.0

listDatabases

public abstract Dataset<Database> listDatabases()

Returns a list of databases available across all sessions.

Returns:

(undocumented)

Since:

2.0.0

listTables

public abstract Dataset<Table> listTables()

Returns a list of tables in the current database. This includes all temporary tables.

Returns:

(undocumented)

Since:

2.0.0

listTables

public abstract Dataset<Table> listTables(java.lang.String dbName)
                                   throws AnalysisException

Returns a list of tables in the specified database. This includes all temporary tables.

Parameters:

dbName - (undocumented)

Returns:

(undocumented)

Throws:

AnalysisException

Since:

2.0.0

listFunctions

public abstract Dataset<Function> listFunctions()

Returns a list of functions registered in the current database. This includes all temporary functions

Returns:

(undocumented)

Since:

2.0.0

listFunctions

public abstract Dataset<Function> listFunctions(java.lang.String dbName)
                                         throws AnalysisException

Returns a list of functions registered in the specified database. This includes all temporary functions

Parameters:

dbName - (undocumented)

Returns:

(undocumented)

Throws:

AnalysisException

Since:

2.0.0

listColumns

public abstract Dataset<Column> listColumns(java.lang.String tableName)
                                     throws AnalysisException

Returns a list of columns for the given table in the current database.

Parameters:

tableName - (undocumented)

Returns:

(undocumented)

Throws:

AnalysisException

Since:

2.0.0

listColumns

public abstract Dataset<Column> listColumns(java.lang.String dbName,
                          java.lang.String tableName)
                                     throws AnalysisException

Returns a list of columns for the given table in the specified database.

Parameters:

dbName - (undocumented)

tableName - (undocumented)

Returns:

(undocumented)

Throws:

AnalysisException

Since:

2.0.0

createExternalTable

public abstract Dataset<Row> createExternalTable(java.lang.String tableName,
                               java.lang.String path)

:: Experimental :: Creates an external table from the given path and returns the corresponding DataFrame. It will use the default data source configured by spark.sql.sources.default.

Parameters:

tableName - (undocumented)

path - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

createExternalTable

public abstract Dataset<Row> createExternalTable(java.lang.String tableName,
                               java.lang.String path,
                               java.lang.String source)

:: Experimental :: Creates an external table from the given path based on a data source and returns the corresponding DataFrame.

Parameters:

tableName - (undocumented)

path - (undocumented)

source - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

createExternalTable

public abstract Dataset<Row> createExternalTable(java.lang.String tableName,
                               java.lang.String source,
                               java.util.Map<java.lang.String,java.lang.String> options)

:: Experimental :: Creates an external table from the given path based on a data source and a set of options. Then, returns the corresponding DataFrame.

Parameters:

tableName - (undocumented)

source - (undocumented)

options - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

createExternalTable

public abstract Dataset<Row> createExternalTable(java.lang.String tableName,
                               java.lang.String source,
                               scala.collection.immutable.Map<java.lang.String,java.lang.String> options)

:: Experimental :: (Scala-specific) Creates an external table from the given path based on a data source and a set of options. Then, returns the corresponding DataFrame.

Parameters:

tableName - (undocumented)

source - (undocumented)

options - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

createExternalTable

public abstract Dataset<Row> createExternalTable(java.lang.String tableName,
                               java.lang.String source,
                               StructType schema,
                               java.util.Map<java.lang.String,java.lang.String> options)

:: Experimental :: Create an external table from the given path based on a data source, a schema and a set of options. Then, returns the corresponding DataFrame.

Parameters:

tableName - (undocumented)

source - (undocumented)

schema - (undocumented)

options - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

createExternalTable

public abstract Dataset<Row> createExternalTable(java.lang.String tableName,
                               java.lang.String source,
                               StructType schema,
                               scala.collection.immutable.Map<java.lang.String,java.lang.String> options)

:: Experimental :: (Scala-specific) Create an external table from the given path based on a data source, a schema and a set of options. Then, returns the corresponding DataFrame.

Parameters:

tableName - (undocumented)

source - (undocumented)

schema - (undocumented)

options - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

dropTempView

public abstract void dropTempView(java.lang.String viewName)

Drops the temporary view with the given view name in the catalog. If the view has been cached before, then it will also be uncached.

Parameters:

viewName - the name of the view to be dropped.

Since:

2.0.0

isCached

public abstract boolean isCached(java.lang.String tableName)

Returns true if the table is currently cached in-memory.

Parameters:

tableName - (undocumented)

Returns:

(undocumented)

Since:

2.0.0

cacheTable

public abstract void cacheTable(java.lang.String tableName)

Caches the specified table in-memory.

Parameters:

tableName - (undocumented)

Since:

2.0.0

uncacheTable

public abstract void uncacheTable(java.lang.String tableName)

Removes the specified table from the in-memory cache.

Parameters:

tableName - (undocumented)

Since:

2.0.0

clearCache

public abstract void clearCache()

Removes all cached tables from the in-memory cache.

Since:

2.0.0