org.apache.spark.sql.sources.v2.writer

Interface SupportsWriteInternalRow

All Superinterfaces:

DataSourceWriter

@InterfaceStability.Unstable
public interface SupportsWriteInternalRow
extends DataSourceWriter

A mix-in interface for DataSourceWriter. Data source writers can implement this interface to write InternalRow directly and avoid the row conversion at Spark side. This is an experimental and unstable interface, as InternalRow is not public and may get changed in the future Spark versions.

Method Summary

Modifier and Type	Method and Description
DataWriterFactory<org.apache.spark.sql.catalyst.InternalRow>	createInternalRowWriterFactory()
default DataWriterFactory<Row>	createWriterFactory() Creates a writer factory which will be serialized and sent to executors.

Methods inherited from interface org.apache.spark.sql.sources.v2.writer.DataSourceWriter

abort, commit

Method Detail

createWriterFactory

default DataWriterFactory<Row> createWriterFactory()

Description copied from interface: DataSourceWriter

Creates a writer factory which will be serialized and sent to executors. If this method fails (by throwing an exception), the action would fail and no Spark job was submitted.

Specified by:

createWriterFactory in interface DataSourceWriter

createInternalRowWriterFactory

DataWriterFactory<org.apache.spark.sql.catalyst.InternalRow> createInternalRowWriterFactory()