trait SupportsMetadataColumns extends Table
An interface for exposing data columns for a table that are not in the table schema. For example, a file source could expose a "file" column that contains the path of the file that contained each row.
The columns returned by #metadataColumns() may be passed as StructField in
requested projections. Sources that implement this interface and column projection using
SupportsPushDownRequiredColumns must accept metadata fields passed to
SupportsPushDownRequiredColumns#pruneColumns(StructType).
If a table column and a metadata column have the same name, the conflict is resolved by either
renaming or suppressing the metadata column. See canRenameConflictingMetadataColumns.
- Annotations
- @Evolving()
- Source
- SupportsMetadataColumns.java
- Since
3.1.0
- Alphabetic
- By Inheritance
- SupportsMetadataColumns
- Table
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def capabilities(): Set[TableCapability]
Returns the set of capabilities for this table.
Returns the set of capabilities for this table.
- Definition Classes
- Table
- abstract def metadataColumns(): Array[MetadataColumn]
Metadata columns that are supported by this
Table.Metadata columns that are supported by this
Table.The columns returned by this method may be passed as
StructFieldin requested projections usingSupportsPushDownRequiredColumns#pruneColumns(StructType).If a table column and a metadata column have the same name, the conflict is resolved by either renaming or suppressing the metadata column. See
canRenameConflictingMetadataColumns.- returns
an array of
MetadataColumn
- abstract def name(): String
A name to identify this table.
A name to identify this table. Implementations should provide a meaningful name, like the database and table name from catalog, or the location of files for this table.
- Definition Classes
- Table
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def canRenameConflictingMetadataColumns(): Boolean
Determines how this data source handles name conflicts between metadata and data columns.
Determines how this data source handles name conflicts between metadata and data columns.
If true, spark will automatically rename the metadata column to resolve the conflict. End users can reliably select metadata columns (renamed or not) with
Dataset.metadataColumn, and internal code can useMetadataAttributeWithLogicalNameto extract the logical name from a metadata attribute.If false, the data column will hide the metadata column. It is recommended that Table implementations which do not support renaming should reject data column names that conflict with metadata column names.
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def columns(): Array[Column]
Returns the columns of this table.
Returns the columns of this table. If the table is not readable and doesn't have a schema, an empty array can be returned here.
- Definition Classes
- Table
- def constraints(): Array[Constraint]
Returns the constraints for this table.
Returns the constraints for this table.
- Definition Classes
- Table
- def currentVersion(): String
Returns the current table version if implementation supports versioning.
Returns the current table version if implementation supports versioning. If the table is not versioned, null can be returned here.
- Definition Classes
- Table
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def partitioning(): Array[Transform]
Returns the physical partitioning of this table.
Returns the physical partitioning of this table.
- Definition Classes
- Table
- def properties(): Map[String, String]
Returns the string map of table properties.
Returns the string map of table properties.
- Definition Classes
- Table
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)
- def schema(): StructType
Returns the schema of this table.
Returns the schema of this table. If the table is not readable and doesn't have a schema, an empty schema can be returned here.
- Definition Classes
- Table
- Annotations
- @Deprecated
- Deprecated
(Since version 3.4.0)