pyspark.sql.functions.kll_sketch_get_rank_bigint#

pyspark.sql.functions.kll_sketch_get_rank_bigint(sketch, quantile)[source]#

Extracts a rank value from a KLL bigint sketch given an input quantile value. The quantile can be a single value or an array.

Added in version 4.1.0.

Parameters:
sketchColumn or column name

The KLL bigint sketch binary representation. A column that evaluates to a binary.

quantileColumn or column name

The quantile value(s) to lookup. A column that evaluates to a long or array. Must be a constant.

Returns:
Column

The rank value(s) (between 0.0 and 1.0). Returns a column that evaluates to a double, or an array of doubles if the quantile argument is an array.

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([1,2,3,4,5], "INT")
>>> sketch_df = df.agg(sf.kll_sketch_agg_bigint("value").alias("sketch"))
>>> sketch_df.select(sf.kll_sketch_get_rank_bigint("sketch", sf.lit(3))).show()
+-------------------------------------+
|kll_sketch_get_rank_bigint(sketch, 3)|
+-------------------------------------+
|                                  0.6|
+-------------------------------------+