Package org.apache.spark.sql.connector.catalog

Class V2TableUtil

Object

org.apache.spark.sql.connector.catalog.V2TableUtil

public class V2TableUtil
extends Object

Constructor Summary

Constructors

Constructor	Description
V2TableUtil()

Method Summary

Modifier and Type	Method	Description
static org.apache.spark.sql.internal.SQLConf	conf()
static scala.collection.immutable.Seq<MetadataColumn>	extractMetadataColumns(org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation relation)	Extracts original column info for all metadata attributes in the relation.
static String	toQualifiedName(CatalogPlugin catalog, Identifier ident)
static scala.collection.immutable.Seq<String>	validateCapturedColumns(Table table, org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation relation, SchemaValidationMode mode)	Validates that captured data columns match the current table schema.
static scala.collection.immutable.Seq<String>	validateCapturedColumns(Table table, scala.collection.immutable.Seq<Column> originCols, SchemaValidationMode mode)	Validates that captured data columns match the current table schema.
static scala.collection.immutable.Seq<String>	validateCapturedMetadataColumns(Table table, org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation relation, SchemaValidationMode mode)	Validates that captured metadata columns are consistent with the current table metadata.
static scala.collection.immutable.Seq<String>	validateCapturedMetadataColumns(Table table, scala.collection.immutable.Seq<MetadataColumn> originMetaCols, SchemaValidationMode mode)	Validates that captured metadata columns are consistent with the current table metadata.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

V2TableUtil

public V2TableUtil()

Method Details

toQualifiedName

public static String toQualifiedName(CatalogPlugin catalog,
 Identifier ident)

validateCapturedColumns

public static scala.collection.immutable.Seq<String> validateCapturedColumns(Table table,
 org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation relation,
 SchemaValidationMode mode)

Validates that captured data columns match the current table schema.

Parameters:

table - the current table metadata

relation - the relation with captured columns

mode - validation mode that defines what changes are acceptable

Returns:

validation errors, or empty sequence if valid

validateCapturedColumns

public static scala.collection.immutable.Seq<String> validateCapturedColumns(Table table,
 scala.collection.immutable.Seq<Column> originCols,
 SchemaValidationMode mode)

Validates that captured data columns match the current table schema.

Checks for: - Column type or nullability changes - Removed columns (missing from the current table schema) - Added columns (new in the current table schema)

Parameters:

table - the current table metadata

originCols - the originally captured columns

mode - validation mode that defines what changes are acceptable

Returns:

validation errors, or empty sequence if valid

validateCapturedMetadataColumns

public static scala.collection.immutable.Seq<String> validateCapturedMetadataColumns(Table table,
 org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation relation,
 SchemaValidationMode mode)

Validates that captured metadata columns are consistent with the current table metadata.

Parameters:

table - the current table metadata

relation - the relation with captured metadata columns

mode - validation mode that defines what changes are acceptable

Returns:

validation errors, or empty sequence if valid

extractMetadataColumns

public static scala.collection.immutable.Seq<MetadataColumn> extractMetadataColumns(org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation relation)

Extracts original column info for all metadata attributes in the relation.

Parameters:

relation - the relation with captured metadata columns

Returns:

metadata columns captured by the relation

validateCapturedMetadataColumns

public static scala.collection.immutable.Seq<String> validateCapturedMetadataColumns(Table table,
 scala.collection.immutable.Seq<MetadataColumn> originMetaCols,
 SchemaValidationMode mode)

Validates that captured metadata columns are consistent with the current table metadata.

Checks for: - Metadata column type or nullability changes - Removed metadata columns (missing from current table)

Parameters:

table - the current table metadata

originMetaCols - the originally captured metadata columns

mode - validation mode that defines what changes are acceptable

Returns:

validation errors, or empty sequence if valid

conf

public static org.apache.spark.sql.internal.SQLConf conf()