pyspark.sql.functions.kll_sketch_merge_bigint#

pyspark.sql.functions.kll_sketch_merge_bigint(left, right)[source]#

Merges two KLL bigint sketch buffers together into one.

Added in version 4.1.0.

Parameters:
leftColumn or column name

The first KLL bigint sketch. A column that evaluates to a binary.

rightColumn or column name

The second KLL bigint sketch. A column that evaluates to a binary.

Returns:
Column

The merged KLL sketch. Returns a column that evaluates to a binary.

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([1,2,3,4,5], "INT")
>>> sketch_df = df.agg(sf.kll_sketch_agg_bigint("value").alias("sketch"))
>>> result = sketch_df.select(sf.kll_sketch_merge_bigint("sketch", "sketch")).first()[0]
>>> result is not None and len(result) > 0
True