org.apache.spark.sql.hive

Class HiveMetastoreCatalog

Object

org.apache.spark.sql.hive.HiveMetastoreCatalog

All Implemented Interfaces:

Logging, org.apache.spark.sql.catalyst.analysis.Catalog

public class HiveMetastoreCatalog
extends Object
implements org.apache.spark.sql.catalyst.analysis.Catalog, Logging

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	HiveMetastoreCatalog.CreateTables
class	HiveMetastoreCatalog.ParquetConversions
class	HiveMetastoreCatalog.PreInsertionCasts
class	HiveMetastoreCatalog.QualifiedTableName A fully qualified identifier for a table (i.e., database.tableName)

Constructor Summary

Constructors

Constructor and Description
HiveMetastoreCatalog(HiveContext hive)

Method Summary

Methods

Modifier and Type	Method and Description
boolean	caseSensitive()
void	createDataSourceTable(String tableName, scala.Option<org.apache.spark.sql.types.StructType> userSpecifiedSchema, String provider, scala.collection.immutable.Map<String,String> options, boolean isExternal) Creates a data source table (a table created with USING clause) in Hive's metastore.
void	createTable(String databaseName, String tableName, scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> schema, boolean allowExisting, scala.Option<org.apache.hadoop.hive.ql.plan.CreateTableDesc> desc) Create table with specified database, table name, table description and schema
org.apache.spark.sql.hive.HiveMetastoreCatalog.CreateTables$	CreateTables()
scala.collection.Seq<scala.Tuple2<String,Object>>	getTables(scala.Option<String> databaseName)
String	hiveDefaultTableFilePath(String tableName)
void	invalidateTable(String databaseName, String tableName)
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan	lookupRelation(scala.collection.Seq<String> tableIdentifier, scala.Option<String> alias)
org.apache.spark.sql.hive.HiveMetastoreCatalog.ParquetConversions$	ParquetConversions()
org.apache.spark.sql.hive.HiveMetastoreCatalog.PreInsertionCasts$	PreInsertionCasts()
void	refreshTable(String databaseName, String tableName)
void	registerTable(scala.collection.Seq<String> tableIdentifier, org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan) UNIMPLEMENTED: It needs to be decided how we will persist in-memory tables to the metastore.
boolean	tableExists(scala.collection.Seq<String> tableIdentifier)
void	unregisterAllTables()
void	unregisterTable(scala.collection.Seq<String> tableIdentifier) UNIMPLEMENTED: It needs to be decided how we will persist in-memory tables to the metastore.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.catalyst.analysis.Catalog

getDBTable, getDbTableName, lookupRelation$default$2, processTableIdentifier

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

HiveMetastoreCatalog

public HiveMetastoreCatalog(HiveContext hive)

Method Detail

refreshTable

public void refreshTable(String databaseName,
                String tableName)

Specified by:

refreshTable in interface org.apache.spark.sql.catalyst.analysis.Catalog

invalidateTable

public void invalidateTable(String databaseName,
                   String tableName)

caseSensitive

public boolean caseSensitive()

Specified by:

caseSensitive in interface org.apache.spark.sql.catalyst.analysis.Catalog

createDataSourceTable

public void createDataSourceTable(String tableName,
                         scala.Option<org.apache.spark.sql.types.StructType> userSpecifiedSchema,
                         String provider,
                         scala.collection.immutable.Map<String,String> options,
                         boolean isExternal)

Creates a data source table (a table created with USING clause) in Hive's metastore. Returns true when the table has been created. Otherwise, false.

hiveDefaultTableFilePath

public String hiveDefaultTableFilePath(String tableName)

tableExists

public boolean tableExists(scala.collection.Seq<String> tableIdentifier)

Specified by:

tableExists in interface org.apache.spark.sql.catalyst.analysis.Catalog

lookupRelation

public org.apache.spark.sql.catalyst.plans.logical.LogicalPlan lookupRelation(scala.collection.Seq<String> tableIdentifier,
                                                                     scala.Option<String> alias)

Specified by:

lookupRelation in interface org.apache.spark.sql.catalyst.analysis.Catalog

getTables

public scala.collection.Seq<scala.Tuple2<String,Object>> getTables(scala.Option<String> databaseName)

Specified by:

getTables in interface org.apache.spark.sql.catalyst.analysis.Catalog

createTable

public void createTable(String databaseName,
               String tableName,
               scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> schema,
               boolean allowExisting,
               scala.Option<org.apache.hadoop.hive.ql.plan.CreateTableDesc> desc)

Create table with specified database, table name, table description and schema

Parameters:

databaseName - Database Name

tableName - Table Name

schema - Schema of the new table, if not specified, will use the schema specified in crtTbl

allowExisting - if true, ignore AlreadyExistsException

desc - CreateTableDesc object which contains the SerDe info. Currently we support most of the features except the bucket.

ParquetConversions

public org.apache.spark.sql.hive.HiveMetastoreCatalog.ParquetConversions$ ParquetConversions()

CreateTables

public org.apache.spark.sql.hive.HiveMetastoreCatalog.CreateTables$ CreateTables()

PreInsertionCasts

public org.apache.spark.sql.hive.HiveMetastoreCatalog.PreInsertionCasts$ PreInsertionCasts()

registerTable

public void registerTable(scala.collection.Seq<String> tableIdentifier,
                 org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)

UNIMPLEMENTED: It needs to be decided how we will persist in-memory tables to the metastore. For now, if this functionality is desired mix in the in-memory OverrideCatalog.

Specified by:

registerTable in interface org.apache.spark.sql.catalyst.analysis.Catalog

unregisterTable

public void unregisterTable(scala.collection.Seq<String> tableIdentifier)

UNIMPLEMENTED: It needs to be decided how we will persist in-memory tables to the metastore. For now, if this functionality is desired mix in the in-memory OverrideCatalog.

Specified by:

unregisterTable in interface org.apache.spark.sql.catalyst.analysis.Catalog

unregisterAllTables

public void unregisterAllTables()

Specified by:

unregisterAllTables in interface org.apache.spark.sql.catalyst.analysis.Catalog