trait SupportsSchemaEvolution extends Table
A mix-in interface for Table schema evolution support. Data sources can implement this
interface to indicate the schema changes they support during write operations. Tables must
report capability TableCapability#AUTOMATIC_SCHEMA_EVOLUTION for schema evolution to
happen.
During automatic schema evolution, Spark computes the set of column changes (e.g. adding a new
column, widening a column type) needed to make the target table schema compatible with the
source data schema. Each candidate change is passed to #supportsColumnChange to
determine whether the data source can apply it. Changes that are not supported are skipped and
Spark will attempt to resolve such changes using casts. If casting is not supported, the query
will fail.
- Annotations
- @Experimental()
- Source
- SupportsSchemaEvolution.java
- Since
4.2.0
- Alphabetic
- By Inheritance
- SupportsSchemaEvolution
- Table
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def capabilities(): Set[TableCapability]
Returns the set of capabilities for this table.
Returns the set of capabilities for this table.
- Definition Classes
- Table
- abstract def name(): String
A name to identify this table.
A name to identify this table. Implementations should provide a meaningful name, like the database and table name from catalog, or the location of files for this table.
- Definition Classes
- Table
- abstract def supportsColumnChange(change: ColumnChange): Boolean
Returns whether the given column change can be applied to the table during automatic schema evolution.
Returns whether the given column change can be applied to the table during automatic schema evolution. This should return
truewhen the data source can physically apply the change, for example adding a new column or performing a lossless type widening.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def columns(): Array[Column]
Returns the columns of this table.
Returns the columns of this table. If the table is not readable and doesn't have a schema, an empty array can be returned here.
- Definition Classes
- Table
- def constraints(): Array[Constraint]
Returns the constraints for this table.
Returns the constraints for this table.
- Definition Classes
- Table
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def id(): String
An ID of the table that can be used to reliably check if two table objects refer to the same metastore entity.
An ID of the table that can be used to reliably check if two table objects refer to the same metastore entity. If a table is dropped and recreated again with the same name, the new table ID must be different. This method must return null if connectors don't support the notion of table ID.
- Definition Classes
- Table
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def partitioning(): Array[Transform]
Returns the physical partitioning of this table.
Returns the physical partitioning of this table.
- Definition Classes
- Table
- def properties(): Map[String, String]
Returns the string map of table properties.
Returns the string map of table properties.
- Definition Classes
- Table
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def version(): String
Returns the version of this table if versioning is supported, null otherwise.
Returns the version of this table if versioning is supported, null otherwise.
This method must not trigger a refresh of the table metadata. It should return the version that corresponds to the current state of this table instance.
- Definition Classes
- Table
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)
- def schema(): StructType
Returns the schema of this table.
Returns the schema of this table. If the table is not readable and doesn't have a schema, an empty schema can be returned here.
- Definition Classes
- Table
- Annotations
- @Deprecated
- Deprecated
(Since version 3.4.0)