Interface RelationCatalog
- All Superinterfaces:
CatalogPlugin,TableCatalog,ViewCatalog
Connectors that expose both tables and views must implement RelationCatalog;
implementing TableCatalog and ViewCatalog directly without
RelationCatalog is rejected at catalog initialization. Connectors that expose only
tables implement just TableCatalog; connectors that expose only views implement just
ViewCatalog; this interface is not relevant to them.
Two principles
ARelationCatalog follows two rules that, taken together, define every cross-cutting
subtlety:
- Orthogonal interfaces. Every
TableCatalogmethod behaves as if views did not exist, and everyViewCatalogmethod behaves as if tables did not exist. From the perspective of aTableCatalogcaller, a view at an identifier is indistinguishable from "nothing there"; symmetrically forViewCatalogon tables. The implementation, of course, knows about both kinds -- it just filters them apart at each method boundary. - Single identifier namespace. Tables and views share one keyspace within a
namespace; the same
Identifiercannot resolve to both at the same time. The implementation typically enforces this with a single backing keyspace plus a kind discriminator.
Per-method cross-type behavior
Active rejection (write-side methods that throw on cross-type collision):Single-RPC perf entry points
The orthogonalTableCatalog and ViewCatalog answer two cross-cutting
questions in two round trips each. RelationCatalog adds dedicated methods so a
catalog can answer both in one round trip:
loadRelation(Identifier)-- the resolver's per-identifier read path. Returns aTablefor a table or aViewfor a view; callers discriminate viainstanceof. Saves theloadTable->loadViewfallback on a cold cache.listRelationSummaries(String[])-- a unified listing of tables and views with the kind preserved on eachTableSummary. Default impl performs bothTableCatalog.listTableSummaries(java.lang.String[])andViewCatalog.listViews(java.lang.String[]); override to fetch in one round trip.
- Since:
- 4.2.0
-
Field Summary
Fields inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog
OPTION_PREFIX, PROP_COLLATION, PROP_COMMENT, PROP_EXTERNAL, PROP_IS_MANAGED_LOCATION, PROP_LOCATION, PROP_OWNER, PROP_PROVIDER, PROP_TABLE_TYPE -
Method Summary
Modifier and TypeMethodDescriptiondefault TableSummary[]listRelationSummaries(String[] namespace) List the tables and views in a namespace, returned asTableSummaryentries with the kind preserved on each summary.loadRelation(Identifier ident) Load the relation for an identifier that may resolve to either a table or a view.default TableloadTable(Identifier ident) Load table metadata byidentifierfrom the catalog.default ViewloadView(Identifier ident) Load view metadata by identifier.default booleantableExists(Identifier ident) Test whether a table exists using anidentifierfrom the catalog.default booleanviewExists(Identifier ident) Test whether a view exists.Methods inherited from interface org.apache.spark.sql.connector.catalog.CatalogPlugin
defaultNamespace, initialize, nameMethods inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog
alterTable, capabilities, changelogStateOptionKeys, createTable, createTable, createTable, createTableLike, dropTable, invalidateTable, listTables, listTableSummaries, loadChangelog, loadTable, loadTable, loadTable, loadTable, purgeTable, renameTable, tableStateOptionKeys, useNullableQuerySchemaMethods inherited from interface org.apache.spark.sql.connector.catalog.ViewCatalog
createOrReplaceView, createView, dropView, invalidateView, listViews, renameView, replaceView
-
Method Details
-
loadRelation
Relation loadRelation(Identifier ident) throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException Load the relation for an identifier that may resolve to either a table or a view.Returns a
Tablefor a table or aViewfor a view; callers discriminate viainstanceof Table/instanceof View. This lets the resolver answer in a single RPC instead of falling back fromTableCatalog.loadTable(org.apache.spark.sql.connector.catalog.Identifier)toViewCatalog.loadView(org.apache.spark.sql.connector.catalog.Identifier). -
listRelationSummaries
default TableSummary[] listRelationSummaries(String[] namespace) throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException, org.apache.spark.sql.catalyst.analysis.NoSuchTableException List the tables and views in a namespace, returned asTableSummaryentries with the kind preserved on each summary.The default implementation enumerates via
TableCatalog.listTableSummaries(java.lang.String[])for tables andViewCatalog.listViews(java.lang.String[])for views (two round trips). Catalogs that can fetch the unified listing in a single round trip should override.- Parameters:
namespace- a multi-part namespace- Returns:
- an array of summaries for both tables and views in the namespace
- Throws:
org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException- if the namespace does not exist (optional)org.apache.spark.sql.catalyst.analysis.NoSuchTableException- if a table listed by the underlying enumeration disappears before its summary can be assembled (default impl only)
-
loadTable
default Table loadTable(Identifier ident) throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException Load table metadata byidentifierfrom the catalog.The default implementation derives from
loadRelation(org.apache.spark.sql.connector.catalog.Identifier): aViewis rejected as not-a-table; aTableis returned. Override only if a tables-only path is materially cheaper than the unified one.- Specified by:
loadTablein interfaceTableCatalog- Parameters:
ident- a table identifier- Returns:
- the table's metadata
- Throws:
org.apache.spark.sql.catalyst.analysis.NoSuchTableException- If the table doesn't exist
-
loadView
default View loadView(Identifier ident) throws org.apache.spark.sql.catalyst.analysis.NoSuchViewException Load view metadata by identifier.The default implementation derives from
loadRelation(org.apache.spark.sql.connector.catalog.Identifier): aViewis returned; anything else (table or absent) is surfaced asNoSuchViewException. Override only if a views-only path is materially cheaper than the unified one.- Specified by:
loadViewin interfaceViewCatalog- Parameters:
ident- a view identifier- Returns:
- the view metadata
- Throws:
org.apache.spark.sql.catalyst.analysis.NoSuchViewException- if the view does not exist
-
tableExists
Test whether a table exists using anidentifierfrom the catalog.The default implementation derives from
loadRelation(org.apache.spark.sql.connector.catalog.Identifier): returnstrueonly if the entry exists and is a table. Override only if a cheaper existence-check path exists.- Specified by:
tableExistsin interfaceTableCatalog- Parameters:
ident- a table identifier- Returns:
- true if a table exists at
ident, false otherwise
-
viewExists
Test whether a view exists.The default implementation calls
ViewCatalog.loadView(org.apache.spark.sql.connector.catalog.Identifier)and catchesNoSuchViewException. Catalogs that can answer existence cheaply should override.The default implementation derives from
loadRelation(org.apache.spark.sql.connector.catalog.Identifier): returnstrueonly if the entry exists and is a view. Override only if a cheaper existence-check path exists.- Specified by:
viewExistsin interfaceViewCatalog- Parameters:
ident- a view identifier- Returns:
- true if a view exists at
ident, false otherwise
-