Package org.apache.spark.sql.connector.catalog

Interface SupportsSchemaEvolution

All Superinterfaces:
Table
@Experimental public interface SupportsSchemaEvolution extends Table
A mix-in interface for Table schema evolution support. Data sources can implement this interface to indicate the schema changes they support during write operations. Tables must report capability TableCapability.AUTOMATIC_SCHEMA_EVOLUTION for schema evolution to happen.

During automatic schema evolution, Spark computes the set of column changes (e.g. adding a new column, widening a column type) needed to make the target table schema compatible with the source data schema. Each candidate change is passed to supportsColumnChange(org.apache.spark.sql.connector.catalog.TableChange.ColumnChange) to determine whether the data source can apply it. Changes that are not supported are skipped and Spark will attempt to resolve such changes using casts. If casting is not supported, the query will fail.

Since:
4.2.0

  • Method Details

    • supportsColumnChange

      boolean supportsColumnChange(TableChange.ColumnChange change)
      Returns whether the given column change can be applied to the table during automatic schema evolution. This should return true when the data source can physically apply the change, for example adding a new column or performing a lossless type widening.