org.apache.spark.sql.connector.catalog

Interface SupportsAtomicPartitionManagement

All Superinterfaces:

SupportsPartitionManagement, Table

@Experimental
public interface SupportsAtomicPartitionManagement
extends SupportsPartitionManagement

An atomic partition interface of Table to operate multiple partitions atomically.

These APIs are used to modify table partition or partition metadata, they will change the table data as well. $createPartitions(org.apache.spark.sql.catalyst.InternalRow[], java.util.Map<java.lang.String, java.lang.String>[]): add an array of partitions and any data they contain to the table $dropPartitions(org.apache.spark.sql.catalyst.InternalRow[]): remove an array of partitions and any data they contain from the table

Since:

3.1.0

Method Summary

Modifier and Type	Method and Description
default void	createPartition(org.apache.spark.sql.catalyst.InternalRow ident, java.util.Map<String,String> properties) Create a partition in table.
void	createPartitions(org.apache.spark.sql.catalyst.InternalRow[] idents, java.util.Map<String,String>[] properties) Create an array of partitions atomically in table.
default boolean	dropPartition(org.apache.spark.sql.catalyst.InternalRow ident) Drop a partition from table.
boolean	dropPartitions(org.apache.spark.sql.catalyst.InternalRow[] idents) Drop an array of partitions atomically from table.

Methods inherited from interface org.apache.spark.sql.connector.catalog.SupportsPartitionManagement

listPartitionIdentifiers, loadPartitionMetadata, partitionExists, partitionSchema, replacePartitionMetadata

Methods inherited from interface org.apache.spark.sql.connector.catalog.Table

capabilities, name, partitioning, properties, schema

Method Detail

createPartition

default void createPartition(org.apache.spark.sql.catalyst.InternalRow ident,
                             java.util.Map<String,String> properties)
                      throws org.apache.spark.sql.catalyst.analysis.PartitionAlreadyExistsException,
                             UnsupportedOperationException

Description copied from interface: SupportsPartitionManagement

Create a partition in table.

Specified by:

createPartition in interface SupportsPartitionManagement

Parameters:

ident - a new partition identifier

properties - the metadata of a partition

Throws:

org.apache.spark.sql.catalyst.analysis.PartitionAlreadyExistsException - If a partition already exists for the identifier

UnsupportedOperationException - If partition property is not supported

dropPartition

default boolean dropPartition(org.apache.spark.sql.catalyst.InternalRow ident)

Description copied from interface: SupportsPartitionManagement

Drop a partition from table.

Specified by:

dropPartition in interface SupportsPartitionManagement

Parameters:

ident - a partition identifier

Returns:

true if a partition was deleted, false if no partition exists for the identifier

createPartitions

void createPartitions(org.apache.spark.sql.catalyst.InternalRow[] idents,
                      java.util.Map<String,String>[] properties)
               throws org.apache.spark.sql.catalyst.analysis.PartitionsAlreadyExistException,
                      UnsupportedOperationException

Create an array of partitions atomically in table.

If any partition already exists, the operation of createPartitions need to be safely rolled back.

Parameters:

idents - an array of new partition identifiers

properties - the metadata of the partitions

Throws:

org.apache.spark.sql.catalyst.analysis.PartitionsAlreadyExistException - If any partition already exists for the identifier

UnsupportedOperationException - If partition property is not supported

dropPartitions

boolean dropPartitions(org.apache.spark.sql.catalyst.InternalRow[] idents)

Drop an array of partitions atomically from table.

If any partition doesn't exists, the operation of dropPartitions need to be safely rolled back.

Parameters:

idents - an array of partition identifiers

Returns:

true if partitions were deleted, false if any partition not exists