The central connector interface for Change Data Capture (CDC).

Connectors implement this minimal interface to expose change data. Spark handles post-processing (carry-over removal, update detection, net change computation) based on the properties declared by the connector.

The columns returned by #columns() must include the following metadata columns:

• _change_type (STRING) — the kind of change: insert, delete, update_preimage, or update_postimage
• _commit_version — the commit version containing this change. Must be either LongType or StringType; all other types are rejected. The column's natural ordering (numeric for LongType, lexicographic for StringType) must match commit order, because the netChanges post-processing path sorts rows of a given row identity by this column to determine the first and last events.
• _commit_timestamp (TIMESTAMP) -- the timestamp of the commit. All rows belonging to a single _commit_version must share the same _commit_timestamp. For streaming reads with post-processing enabled, two additional requirements apply:
• All rows of a single commit must appear in the same micro-batch (i.e. micro-batch boundaries align with commit boundaries).
• Each micro-batch's rows must have _commit_timestamp strictly greater than the maximum _commit_timestamp of any prior micro-batch.

Streaming post-processing uses _commit_timestamp as event time with a zero-delay watermark, so once a micro-batch observes max event time T the global watermark advances to T. Both Spark's late-event filter and its state-eviction predicate then use eventTime <= T -- so any later row at _commit_timestamp <= T (whether from the same commit split across batches, a different commit emitted later, or simply an out-of-order commit) is silently dropped as late. Requirement 1 keeps a single commit's rows together; requirement 2 keeps distinct commits in strictly increasing event-time order across batches. Multiple distinct commits with equal _commit_timestamp are allowed within a single micro-batch -- only across batches does timestamp progression need to be strictly increasing. Atomic-commit CDC connectors (e.g. Delta versions, Iceberg snapshots) that derive _commit_timestamp from wall-clock time at commit time naturally satisfy both requirements. _commit_timestamp must be non-NULL on every row of a streaming read engaging post-processing; both the row-level Aggregate path and the netChanges transformWithState path raise CHANGELOG_CONTRACT_VIOLATION.NULL_COMMIT_TIMESTAMP on a violation

Streaming reads support carry-over removal, update detection, and net change computation. Two streaming-specific behaviors to be aware of:

• Output is buffered until the watermark advances past the commit. When a micro-batch ingests a commit, that commit's output rows are buffered in state and not emitted in the same batch. They are emitted by a later micro-batch -- whichever one advances the watermark past the commit's _commit_timestamp. The last commit's output is emitted when the source terminates.
• netChanges only merges changes that are buffered together. When each row identity appears in at most one commit within any buffered window, the streaming output is the same as computeUpdates. Cross-commit merging only happens when several commits touch the same row before the earliest one's output has been released. For full-range collapse, use a batch read.

Pushdown contract. When any post-processing pass applies (carry-over removal, update detection, or netChanges), Spark only pushes predicates that reference _commit_version, _commit_timestamp, or columns named by #rowId() to the connector's org.apache.spark.sql.connector.read.SupportsPushDownFilters / org.apache.spark.sql.connector.read.SupportsPushDownV2Filters. Predicates on _change_type, the #rowVersion() column, or non-rowId data columns are kept above the scan: pushing them would drop one half of a delete/insert pair within a row-identity group and silently break post-processing. Catalyst's pushdown rules enforce this via the rewrite operators, so connectors do not need to code the restriction themselves -- but must not bypass it via connector-specific options. When no post-processing pass applies, Spark does not impose any CDC-specific predicate-pushdown restriction. org.apache.spark.sql.connector.read.SupportsPushDownRequiredColumns (column pruning) is unrestricted in either case: Spark's pruning already respects what the rewrite operators reference.