Package org.apache.spark.sql.connector.catalog

Interface RelationCatalog

All Superinterfaces:

CatalogPlugin, TableCatalog, ViewCatalog

@Evolving
public interface RelationCatalog
extends TableCatalog, ViewCatalog

Catalog API for connectors that expose both tables and views in a single shared identifier namespace.

Connectors that expose both tables and views must implement RelationCatalog; implementing TableCatalog and ViewCatalog directly without RelationCatalog is rejected at catalog initialization. Connectors that expose only tables implement just TableCatalog; connectors that expose only views implement just ViewCatalog; this interface is not relevant to them.

Two principles

A RelationCatalog follows two rules that, taken together, define every cross-cutting subtlety:

1. Orthogonal interfaces. Every TableCatalog method behaves as if views did not exist, and every ViewCatalog method behaves as if tables did not exist. From the perspective of a TableCatalog caller, a view at an identifier is indistinguishable from "nothing there"; symmetrically for ViewCatalog on tables. The implementation, of course, knows about both kinds -- it just filters them apart at each method boundary.
2. Single identifier namespace. Tables and views share one keyspace within a namespace; the same Identifier cannot resolve to both at the same time. The implementation typically enforces this with a single backing keyspace plus a kind discriminator.

Per-method cross-type behavior

Active rejection (write-side methods that throw on cross-type collision):

Cross-type rejection

Method	Rejects when	Throws
TableCatalog.createTable(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.types.StructType, org.apache.spark.sql.connector.expressions.Transform[], java.util.Map<java.lang.String, java.lang.String>)	a view sits at ident	TableAlreadyExistsException
TableCatalog.renameTable(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.connector.catalog.Identifier)	a view sits at newIdent	TableAlreadyExistsException
ViewCatalog.createView(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.connector.catalog.View)	a table sits at ident	ViewAlreadyExistsException
ViewCatalog.createOrReplaceView(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.connector.catalog.View)	a table sits at ident	ViewAlreadyExistsException
ViewCatalog.replaceView(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.connector.catalog.View)	a table sits at ident	NoSuchViewException
ViewCatalog.renameView(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.connector.catalog.Identifier)	a table sits at newIdent	ViewAlreadyExistsException

Passive filtering (read / non-collision mutation methods that behave as if the wrong kind doesn't exist):

Cross-type filtering

Method	On wrong-kind ident
TableCatalog.loadTable(Identifier)	throws NoSuchTableException for a view
TableCatalog.loadTable(Identifier, String) / TableCatalog.loadTable(Identifier, long)	throws NoSuchTableException for a view (no perf opt-in -- time-travel does not apply to views)
TableCatalog.tableExists(org.apache.spark.sql.connector.catalog.Identifier)	returns false for a view
TableCatalog.dropTable(org.apache.spark.sql.connector.catalog.Identifier) / TableCatalog.purgeTable(org.apache.spark.sql.connector.catalog.Identifier)	returns false for a view; does not drop it
TableCatalog.renameTable(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.connector.catalog.Identifier)	throws NoSuchTableException when the source is a view
TableCatalog.listTables(java.lang.String[])	tables only
ViewCatalog.loadView(org.apache.spark.sql.connector.catalog.Identifier)	throws NoSuchViewException for a table
ViewCatalog.viewExists(org.apache.spark.sql.connector.catalog.Identifier)	returns false for a table
ViewCatalog.dropView(org.apache.spark.sql.connector.catalog.Identifier)	returns false for a table; does not drop it
ViewCatalog.renameView(org.apache.spark.sql.connector.catalog.Identifier, org.apache.spark.sql.connector.catalog.Identifier)	throws NoSuchViewException when the source is a table
ViewCatalog.listViews(java.lang.String[])	views only

Single-RPC perf entry points

The orthogonal TableCatalog and ViewCatalog answer two cross-cutting questions in two round trips each. RelationCatalog adds dedicated methods so a catalog can answer both in one round trip:

• loadRelation(Identifier) -- the resolver's per-identifier read path. Returns a Table for a table or a View for a view; callers discriminate via instanceof. Saves the loadTable -> loadView fallback on a cold cache.
• listRelationSummaries(String[]) -- a unified listing of tables and views with the kind preserved on each TableSummary. Default impl performs both TableCatalog.listTableSummaries(java.lang.String[]) and ViewCatalog.listViews(java.lang.String[]); override to fetch in one round trip.

Since:

4.2.0

Field Summary

Fields inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog

OPTION_PREFIX, PROP_COLLATION, PROP_COMMENT, PROP_EXTERNAL, PROP_IS_MANAGED_LOCATION, PROP_LOCATION, PROP_OWNER, PROP_PROVIDER, PROP_TABLE_TYPE

Method Summary

Modifier and Type	Method	Description
default TableSummary[]	listRelationSummaries(String[] namespace)	List the tables and views in a namespace, returned as TableSummary entries with the kind preserved on each summary.
Relation	loadRelation(Identifier ident)	Load the relation for an identifier that may resolve to either a table or a view.
default Table	loadTable(Identifier ident)	Load table metadata by identifier from the catalog.
default View	loadView(Identifier ident)	Load view metadata by identifier.
default boolean	tableExists(Identifier ident)	Test whether a table exists using an identifier from the catalog.
default boolean	viewExists(Identifier ident)	Test whether a view exists.

Methods inherited from interface org.apache.spark.sql.connector.catalog.CatalogPlugin

defaultNamespace, initialize, name

Methods inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog

alterTable, capabilities, createTable, createTable, createTable, createTableLike, dropTable, invalidateTable, listTables, listTableSummaries, loadChangelog, loadTable, loadTable, loadTable, purgeTable, renameTable, useNullableQuerySchema

Methods inherited from interface org.apache.spark.sql.connector.catalog.ViewCatalog

createOrReplaceView, createView, dropView, invalidateView, listViews, renameView, replaceView

Method Details

loadRelation

Relation loadRelation(Identifier ident)
               throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException

Load the relation for an identifier that may resolve to either a table or a view.

Returns a Table for a table or a View for a view; callers discriminate via instanceof Table / instanceof View. This lets the resolver answer in a single RPC instead of falling back from TableCatalog.loadTable(org.apache.spark.sql.connector.catalog.Identifier) to ViewCatalog.loadView(org.apache.spark.sql.connector.catalog.Identifier).

Parameters:

ident - the identifier

Returns:

a Table for tables, or a View for views

Throws:

org.apache.spark.sql.catalyst.analysis.NoSuchTableException - if neither a table nor a view exists at ident

listRelationSummaries

default TableSummary[] listRelationSummaries(String[] namespace)
                                      throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException,
org.apache.spark.sql.catalyst.analysis.NoSuchTableException

List the tables and views in a namespace, returned as TableSummary entries with the kind preserved on each summary.

The default implementation enumerates via TableCatalog.listTableSummaries(java.lang.String[]) for tables and ViewCatalog.listViews(java.lang.String[]) for views (two round trips). Catalogs that can fetch the unified listing in a single round trip should override.

Parameters:

namespace - a multi-part namespace

Returns:

an array of summaries for both tables and views in the namespace

Throws:

org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - if the namespace does not exist (optional)

org.apache.spark.sql.catalyst.analysis.NoSuchTableException - if a table listed by the underlying enumeration disappears before its summary can be assembled (default impl only)

loadTable

default Table loadTable(Identifier ident)
                 throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException

Load table metadata by identifier from the catalog.

The default implementation derives from loadRelation(org.apache.spark.sql.connector.catalog.Identifier): a View is rejected as not-a-table; a Table is returned. Override only if a tables-only path is materially cheaper than the unified one.

Specified by:

loadTable in interface TableCatalog

Parameters:

ident - a table identifier

Returns:

the table's metadata

Throws:

org.apache.spark.sql.catalyst.analysis.NoSuchTableException - If the table doesn't exist

loadView

default View loadView(Identifier ident)
               throws org.apache.spark.sql.catalyst.analysis.NoSuchViewException

Load view metadata by identifier.

The default implementation derives from loadRelation(org.apache.spark.sql.connector.catalog.Identifier): a View is returned; anything else (table or absent) is surfaced as NoSuchViewException. Override only if a views-only path is materially cheaper than the unified one.

Specified by:

loadView in interface ViewCatalog

Parameters:

ident - a view identifier

Returns:

the view metadata

Throws:

org.apache.spark.sql.catalyst.analysis.NoSuchViewException - if the view does not exist

tableExists

default boolean tableExists(Identifier ident)

Test whether a table exists using an identifier from the catalog.

The default implementation derives from loadRelation(org.apache.spark.sql.connector.catalog.Identifier): returns true only if the entry exists and is a table. Override only if a cheaper existence-check path exists.

Specified by:

tableExists in interface TableCatalog

Parameters:

ident - a table identifier

Returns:

true if a table exists at ident, false otherwise

viewExists

default boolean viewExists(Identifier ident)

Test whether a view exists.

The default implementation calls ViewCatalog.loadView(org.apache.spark.sql.connector.catalog.Identifier) and catches NoSuchViewException. Catalogs that can answer existence cheaply should override.

The default implementation derives from loadRelation(org.apache.spark.sql.connector.catalog.Identifier): returns true only if the entry exists and is a view. Override only if a cheaper existence-check path exists.

Specified by:

viewExists in interface ViewCatalog

Parameters:

ident - a view identifier

Returns:

true if a view exists at ident, false otherwise