Enum Class TableCapability
- All Implemented Interfaces:
Serializable,Comparable<TableCapability>,Constable
Table implementation.
Tables use Table.capabilities() to return a set of capabilities. Each capability signals
to Spark that the table supports a feature identified by the capability. For example, returning
BATCH_READ allows Spark to read from the table using a batch scan.
- Since:
- 3.0.0
-
Nested Class Summary
Nested classes/interfaces inherited from class java.lang.Enum
Enum.EnumDesc<E extends Enum<E>> -
Enum Constant Summary
Enum ConstantsEnum ConstantDescriptionSignals that the table accepts input of any schema in a write operation.Signals that table supports Spark altering the schema if necessary as part of an operation.Signals that the table supports reads in batch execution mode.Signals that the table supports append writes in batch execution mode.Signals that the table supports reads in continuous streaming execution mode.Signals that the table wants Spark to auto-fill generated column values and enforce generated column constraints during writes.Signals that the table supports reads in micro-batch streaming execution mode.Signals that the table can replace existing data that matches a filter with appended data in a write operation.Signals that the table can dynamically replace existing data partitions with appended data in a write operation.Signals that Spark may fuse two batch scans of this table that differ only in their projected columns and/or pushed filters into a single scan (Spark-side scan merging).Signals that the table supports append writes in streaming execution mode.Signals that the table can be truncated in a write operation.Signals that the table supports append writes using the V1 InsertableRelation interface. -
Method Summary
Modifier and TypeMethodDescriptionstatic TableCapabilityReturns the enum constant of this class with the specified name.static TableCapability[]values()Returns an array containing the constants of this enum class, in the order they are declared.Methods inherited from class java.lang.Enum
compareTo, describeConstable, equals, getDeclaringClass, hashCode, name, ordinal, toString, valueOf
-
Enum Constant Details
-
BATCH_READ
Signals that the table supports reads in batch execution mode. -
MICRO_BATCH_READ
Signals that the table supports reads in micro-batch streaming execution mode. -
CONTINUOUS_READ
Signals that the table supports reads in continuous streaming execution mode. -
BATCH_WRITE
Signals that the table supports append writes in batch execution mode.Tables that return this capability must support appending data and may also support additional write modes, like
TRUNCATE,OVERWRITE_BY_FILTER, andOVERWRITE_DYNAMIC. -
STREAMING_WRITE
Signals that the table supports append writes in streaming execution mode.Tables that return this capability must support appending data and may also support additional write modes, like
TRUNCATE,OVERWRITE_BY_FILTER, andOVERWRITE_DYNAMIC. -
TRUNCATE
Signals that the table can be truncated in a write operation.Truncating a table removes all existing rows.
See
SupportsTruncate. -
OVERWRITE_BY_FILTER
Signals that the table can replace existing data that matches a filter with appended data in a write operation.See
SupportsOverwriteV2. -
OVERWRITE_DYNAMIC
Signals that the table can dynamically replace existing data partitions with appended data in a write operation. -
ACCEPT_ANY_SCHEMA
Signals that the table accepts input of any schema in a write operation. -
AUTOMATIC_SCHEMA_EVOLUTION
Signals that table supports Spark altering the schema if necessary as part of an operation. -
V1_BATCH_WRITE
Signals that the table supports append writes using the V1 InsertableRelation interface.Tables that return this capability must create a V1Write and may also support additional write modes, like
TRUNCATE, andOVERWRITE_BY_FILTER, but cannot supportOVERWRITE_DYNAMIC. -
GENERATE_COLUMN_VALUES_ON_WRITE
Signals that the table wants Spark to auto-fill generated column values and enforce generated column constraints during writes.When this capability is present, Spark will:
- Auto-compute missing generated column values using the generation expression for by-name writes. Ordinary by-position writes still require the input to provide a value for every table column.
- Validate explicitly-provided generated column values against the generation expression.
Without this capability, the connector is responsible for handling generated column values during writes.
- Since:
- 4.3.0
-
SCAN_MERGING
Signals that Spark may fuse two batch scans of this table that differ only in their projected columns and/or pushed filters into a single scan (Spark-side scan merging).By returning this capability a table declares a determinism contract: holding the scan options constant, the rows and columns a scan reads are fully determined by the filters pushed via
SupportsPushDownV2Filtersand the columns pruned viaSupportsPushDownRequiredColumns. Equivalently, obtaining a freshScanBuilderwith the same options and re-applying the same pushed filters and pruned columns yields an equivalent scan.Given that contract, Spark builds the merged scan itself: it prunes a fresh ScanBuilder to the union of both read schemas, re-pushes the (possibly OR-widened) filters, and builds. The merged scan reads the union of the two scans' columns and a superset of their rows; each original scan's result is recovered by a projection and filter applied above it. The connector supplies no merge logic of its own.
This capability lives on the table rather than on the scan so that a source using the V1 scan fallback (whose scan Spark wraps in an internal wrapper) can still opt in. A table need not reason about pushdowns that are not reproducible this way (a pushed aggregate, join, variant extraction, limit, offset, top-N, or table sample): Spark tracks those on its own side while building the scan and never merges a scan that carries one, whether or not the table returns this capability.
- Since:
- 4.3.0
-
-
Method Details
-
values
Returns an array containing the constants of this enum class, in the order they are declared.- Returns:
- an array containing the constants of this enum class, in the order they are declared
-
valueOf
Returns the enum constant of this class with the specified name. The string must match exactly an identifier used to declare an enum constant in this class. (Extraneous whitespace characters are not permitted.)- Parameters:
name- the name of the enum constant to be returned.- Returns:
- the enum constant with the specified name
- Throws:
IllegalArgumentException- if this enum class has no constant with the specified nameNullPointerException- if the argument is null
-