JavaPairReceiverInputDStream (Spark 2.3.2 JavaDoc)

Object
- org.apache.spark.streaming.api.java.JavaPairDStream<K,V>
- - org.apache.spark.streaming.api.java.JavaPairInputDStream<K,V>
  - - org.apache.spark.streaming.api.java.JavaPairReceiverInputDStream<K,V>

All Implemented Interfaces:

java.io.Serializable, JavaDStreamLike<scala.Tuple2<K,V>,JavaPairDStream<K,V>,JavaPairRDD<K,V>>
```
public class JavaPairReceiverInputDStream<K,V>
extends JavaPairInputDStream<K,V>
```
A Java-friendly interface to ReceiverInputDStream, the abstract class for defining any input stream that receives data over the network.

See Also:

Serialized Form

Constructor Summary

Constructors
Constructor and Description
`JavaPairReceiverInputDStream(ReceiverInputDStream<scala.Tuple2<K,V>> receiverInputDStream, scala.reflect.ClassTag<K> kClassTag, scala.reflect.ClassTag<V> vClassTag)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static JavaPairDStream<K,V>`	`cache()`
`static DStream<T>`	`checkpoint(Duration interval)`
`static scala.reflect.ClassTag<scala.Tuple2<K,V>>`	`classTag()`
`static <W> JavaPairDStream<K,scala.Tuple2<Iterable<V>,Iterable<W>>>`	`cogroup(JavaPairDStream<K,W> other)`
`static <W> JavaPairDStream<K,scala.Tuple2<Iterable<V>,Iterable<W>>>`	`cogroup(JavaPairDStream<K,W> other, int numPartitions)`
`static <W> JavaPairDStream<K,scala.Tuple2<Iterable<V>,Iterable<W>>>`	`cogroup(JavaPairDStream<K,W> other, Partitioner partitioner)`
`static <C> JavaPairDStream<K,C>`	`combineByKey(Function<V,C> createCombiner, Function2<C,V,C> mergeValue, Function2<C,C,C> mergeCombiners, Partitioner partitioner)`
`static <C> JavaPairDStream<K,C>`	`combineByKey(Function<V,C> createCombiner, Function2<C,V,C> mergeValue, Function2<C,C,C> mergeCombiners, Partitioner partitioner, boolean mapSideCombine)`
`static JavaPairRDD<K,V>`	`compute(Time validTime)`
`static StreamingContext`	`context()`
`static JavaDStream<Long>`	`count()`
`static JavaPairDStream<T,Long>`	`countByValue()`
`static JavaPairDStream<T,Long>`	`countByValue(int numPartitions)`
`static JavaPairDStream<T,Long>`	`countByValueAndWindow(Duration windowDuration, Duration slideDuration)`
`static JavaPairDStream<T,Long>`	`countByValueAndWindow(Duration windowDuration, Duration slideDuration, int numPartitions)`
`static JavaDStream<Long>`	`countByWindow(Duration windowDuration, Duration slideDuration)`
`static DStream<scala.Tuple2<K,V>>`	`dstream()`
`static JavaPairDStream<K,V>`	`filter(Function<scala.Tuple2<K,V>,Boolean> f)`
`static <U> JavaDStream<U>`	`flatMap(FlatMapFunction<T,U> f)`
`static <K2,V2> JavaPairDStream<K2,V2>`	`flatMapToPair(PairFlatMapFunction<T,K2,V2> f)`
`static <U> JavaPairDStream<K,U>`	`flatMapValues(Function<V,Iterable<U>> f)`
`static void`	`foreachRDD(VoidFunction<R> foreachFunc)`
`static void`	`foreachRDD(VoidFunction2<R,Time> foreachFunc)`
`static <K,V> JavaPairReceiverInputDStream<K,V>`	`fromReceiverInputDStream(ReceiverInputDStream<scala.Tuple2<K,V>> receiverInputDStream, scala.reflect.ClassTag<K> evidence$1, scala.reflect.ClassTag<V> evidence$2)` Convert a scala `ReceiverInputDStream` to a Java-friendly `JavaReceiverInputDStream`.
`static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,Optional<W>>>`	`fullOuterJoin(JavaPairDStream<K,W> other)`
`static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,Optional<W>>>`	`fullOuterJoin(JavaPairDStream<K,W> other, int numPartitions)`
`static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,Optional<W>>>`	`fullOuterJoin(JavaPairDStream<K,W> other, Partitioner partitioner)`
`static JavaDStream<java.util.List<T>>`	`glom()`
`static JavaPairDStream<K,Iterable<V>>`	`groupByKey()`
`static JavaPairDStream<K,Iterable<V>>`	`groupByKey(int numPartitions)`
`static JavaPairDStream<K,Iterable<V>>`	`groupByKey(Partitioner partitioner)`
`static JavaPairDStream<K,Iterable<V>>`	`groupByKeyAndWindow(Duration windowDuration)`
`static JavaPairDStream<K,Iterable<V>>`	`groupByKeyAndWindow(Duration windowDuration, Duration slideDuration)`
`static JavaPairDStream<K,Iterable<V>>`	`groupByKeyAndWindow(Duration windowDuration, Duration slideDuration, int numPartitions)`
`static JavaPairDStream<K,Iterable<V>>`	`groupByKeyAndWindow(Duration windowDuration, Duration slideDuration, Partitioner partitioner)`
`static InputDStream<scala.Tuple2<K,V>>`	`inputDStream()`
`static <W> JavaPairDStream<K,scala.Tuple2<V,W>>`	`join(JavaPairDStream<K,W> other)`
`static <W> JavaPairDStream<K,scala.Tuple2<V,W>>`	`join(JavaPairDStream<K,W> other, int numPartitions)`
`static <W> JavaPairDStream<K,scala.Tuple2<V,W>>`	`join(JavaPairDStream<K,W> other, Partitioner partitioner)`
`scala.reflect.ClassTag<K>`	`kClassTag()`
`static scala.reflect.ClassTag<K>`	`kManifest()`
`static <W> JavaPairDStream<K,scala.Tuple2<V,Optional<W>>>`	`leftOuterJoin(JavaPairDStream<K,W> other)`
`static <W> JavaPairDStream<K,scala.Tuple2<V,Optional<W>>>`	`leftOuterJoin(JavaPairDStream<K,W> other, int numPartitions)`
`static <W> JavaPairDStream<K,scala.Tuple2<V,Optional<W>>>`	`leftOuterJoin(JavaPairDStream<K,W> other, Partitioner partitioner)`
`static <U> JavaDStream<U>`	`map(Function<T,U> f)`
`static <U> JavaDStream<U>`	`mapPartitions(FlatMapFunction<java.util.Iterator<T>,U> f)`
`static <K2,V2> JavaPairDStream<K2,V2>`	`mapPartitionsToPair(PairFlatMapFunction<java.util.Iterator<T>,K2,V2> f)`
`static <K2,V2> JavaPairDStream<K2,V2>`	`mapToPair(PairFunction<T,K2,V2> f)`
`static <U> JavaPairDStream<K,U>`	`mapValues(Function<V,U> f)`
`static <StateType,MappedType> JavaMapWithStateDStream<K,V,StateType,MappedType>`	`mapWithState(StateSpec<K,V,StateType,MappedType> spec)`
`static JavaPairDStream<K,V>`	`persist()`
`static JavaPairDStream<K,V>`	`persist(StorageLevel storageLevel)`
`static void`	`print()`
`static void`	`print(int num)`
`ReceiverInputDStream<scala.Tuple2<K,V>>`	`receiverInputDStream()`
`static JavaDStream<T>`	`reduce(Function2<T,T,T> f)`
`static JavaPairDStream<K,V>`	`reduceByKey(Function2<V,V,V> func)`
`static JavaPairDStream<K,V>`	`reduceByKey(Function2<V,V,V> func, int numPartitions)`
`static JavaPairDStream<K,V>`	`reduceByKey(Function2<V,V,V> func, Partitioner partitioner)`
`static JavaPairDStream<K,V>`	`reduceByKeyAndWindow(Function2<V,V,V> reduceFunc, Duration windowDuration)`
`static JavaPairDStream<K,V>`	`reduceByKeyAndWindow(Function2<V,V,V> reduceFunc, Duration windowDuration, Duration slideDuration)`
`static JavaPairDStream<K,V>`	`reduceByKeyAndWindow(Function2<V,V,V> reduceFunc, Duration windowDuration, Duration slideDuration, int numPartitions)`
`static JavaPairDStream<K,V>`	`reduceByKeyAndWindow(Function2<V,V,V> reduceFunc, Duration windowDuration, Duration slideDuration, Partitioner partitioner)`
`static JavaPairDStream<K,V>`	`reduceByKeyAndWindow(Function2<V,V,V> reduceFunc, Function2<V,V,V> invReduceFunc, Duration windowDuration, Duration slideDuration)`
`static JavaPairDStream<K,V>`	`reduceByKeyAndWindow(Function2<V,V,V> reduceFunc, Function2<V,V,V> invReduceFunc, Duration windowDuration, Duration slideDuration, int numPartitions, Function<scala.Tuple2<K,V>,Boolean> filterFunc)`
`static JavaPairDStream<K,V>`	`reduceByKeyAndWindow(Function2<V,V,V> reduceFunc, Function2<V,V,V> invReduceFunc, Duration windowDuration, Duration slideDuration, Partitioner partitioner, Function<scala.Tuple2<K,V>,Boolean> filterFunc)`
`static JavaDStream<T>`	`reduceByWindow(Function2<T,T,T> reduceFunc, Duration windowDuration, Duration slideDuration)`
`static JavaDStream<T>`	`reduceByWindow(Function2<T,T,T> reduceFunc, Function2<T,T,T> invReduceFunc, Duration windowDuration, Duration slideDuration)`
`static JavaPairDStream<K,V>`	`repartition(int numPartitions)`
`static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,W>>`	`rightOuterJoin(JavaPairDStream<K,W> other)`
`static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,W>>`	`rightOuterJoin(JavaPairDStream<K,W> other, int numPartitions)`
`static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,W>>`	`rightOuterJoin(JavaPairDStream<K,W> other, Partitioner partitioner)`
`static void`	`saveAsHadoopFiles(String prefix, String suffix)`
`static <F extends org.apache.hadoop.mapred.OutputFormat<?,?>> void`	`saveAsHadoopFiles(String prefix, String suffix, Class<?> keyClass, Class<?> valueClass, Class<F> outputFormatClass)`
`static <F extends org.apache.hadoop.mapred.OutputFormat<?,?>> void`	`saveAsHadoopFiles(String prefix, String suffix, Class<?> keyClass, Class<?> valueClass, Class<F> outputFormatClass, org.apache.hadoop.mapred.JobConf conf)`
`static void`	`saveAsNewAPIHadoopFiles(String prefix, String suffix)`
`static <F extends org.apache.hadoop.mapreduce.OutputFormat<?,?>> void`	`saveAsNewAPIHadoopFiles(String prefix, String suffix, Class<?> keyClass, Class<?> valueClass, Class<F> outputFormatClass)`
`static <F extends org.apache.hadoop.mapreduce.OutputFormat<?,?>> void`	`saveAsNewAPIHadoopFiles(String prefix, String suffix, Class<?> keyClass, Class<?> valueClass, Class<F> outputFormatClass, org.apache.hadoop.conf.Configuration conf)`
`static <F extends org.apache.hadoop.mapreduce.OutputFormat<?,?>> org.apache.hadoop.conf.Configuration`	`saveAsNewAPIHadoopFiles$default$6()`
`static JavaDStream<Long>`	`scalaIntToJavaLong(DStream<Object> in)`
`static java.util.List<R>`	`slice(Time fromTime, Time toTime)`
`static JavaDStream<scala.Tuple2<K,V>>`	`toJavaDStream()`
`static <U> JavaDStream<U>`	`transform(Function<R,JavaRDD<U>> transformFunc)`
`static <U> JavaDStream<U>`	`transform(Function2<R,Time,JavaRDD<U>> transformFunc)`
`static <K2,V2> JavaPairDStream<K2,V2>`	`transformToPair(Function<R,JavaPairRDD<K2,V2>> transformFunc)`
`static <K2,V2> JavaPairDStream<K2,V2>`	`transformToPair(Function2<R,Time,JavaPairRDD<K2,V2>> transformFunc)`
`static <U,W> JavaDStream<W>`	`transformWith(JavaDStream<U> other, Function3<R,JavaRDD<U>,Time,JavaRDD<W>> transformFunc)`
`static <K2,V2,W> JavaDStream<W>`	`transformWith(JavaPairDStream<K2,V2> other, Function3<R,JavaPairRDD<K2,V2>,Time,JavaRDD<W>> transformFunc)`
`static <U,K2,V2> JavaPairDStream<K2,V2>`	`transformWithToPair(JavaDStream<U> other, Function3<R,JavaRDD<U>,Time,JavaPairRDD<K2,V2>> transformFunc)`
`static <K2,V2,K3,V3> JavaPairDStream<K3,V3>`	`transformWithToPair(JavaPairDStream<K2,V2> other, Function3<R,JavaPairRDD<K2,V2>,Time,JavaPairRDD<K3,V3>> transformFunc)`
`static JavaPairDStream<K,V>`	`union(JavaPairDStream<K,V> that)`
`static <S> JavaPairDStream<K,S>`	`updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc)`
`static <S> JavaPairDStream<K,S>`	`updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc, int numPartitions)`
`static <S> JavaPairDStream<K,S>`	`updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc, Partitioner partitioner)`
`static <S> JavaPairDStream<K,S>`	`updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc, Partitioner partitioner, JavaPairRDD<K,S> initialRDD)`
`scala.reflect.ClassTag<V>`	`vClassTag()`
`static scala.reflect.ClassTag<V>`	`vManifest()`
`static JavaPairDStream<K,V>`	`window(Duration windowDuration)`
`static JavaPairDStream<K,V>`	`window(Duration windowDuration, Duration slideDuration)`
`static JavaPairRDD<K,V>`	`wrapRDD(RDD<scala.Tuple2<K,V>> rdd)`

Methods inherited from class org.apache.spark.streaming.api.java.JavaPairInputDStream
fromInputDStream, inputDStream

Methods inherited from class org.apache.spark.streaming.api.java.JavaPairDStream
cache, classTag, cogroup, cogroup, cogroup, combineByKey, combineByKey, compute, dstream, filter, flatMapValues, fromJavaDStream, fromPairDStream, fullOuterJoin, fullOuterJoin, fullOuterJoin, groupByKey, groupByKey, groupByKey, groupByKeyAndWindow, groupByKeyAndWindow, groupByKeyAndWindow, groupByKeyAndWindow, join, join, join, kManifest, leftOuterJoin, leftOuterJoin, leftOuterJoin, mapValues, mapWithState, persist, persist, reduceByKey, reduceByKey, reduceByKey, reduceByKeyAndWindow, reduceByKeyAndWindow, reduceByKeyAndWindow, reduceByKeyAndWindow, reduceByKeyAndWindow, reduceByKeyAndWindow, reduceByKeyAndWindow, repartition, rightOuterJoin, rightOuterJoin, rightOuterJoin, saveAsHadoopFiles, saveAsHadoopFiles, saveAsHadoopFiles, saveAsNewAPIHadoopFiles, saveAsNewAPIHadoopFiles, saveAsNewAPIHadoopFiles, scalaToJavaLong, toJavaDStream, union, updateStateByKey, updateStateByKey, updateStateByKey, updateStateByKey, vManifest, window, window, wrapRDD

Methods inherited from class Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.streaming.api.java.JavaDStreamLike
checkpoint, context, count, countByValue, countByValue, countByValueAndWindow, countByValueAndWindow, countByWindow, flatMap, flatMapToPair, foreachRDD, foreachRDD, glom, map, mapPartitions, mapPartitionsToPair, mapToPair, print, print, reduce, reduceByWindow, reduceByWindow, scalaIntToJavaLong, slice, transform, transform, transformToPair, transformToPair, transformWith, transformWith, transformWithToPair, transformWithToPair

Constructor Detail

JavaPairReceiverInputDStream

public JavaPairReceiverInputDStream(ReceiverInputDStream<scala.Tuple2<K,V>> receiverInputDStream,
                                    scala.reflect.ClassTag<K> kClassTag,
                                    scala.reflect.ClassTag<V> vClassTag)

Method Detail

fromReceiverInputDStream

public static <K,V> JavaPairReceiverInputDStream<K,V> fromReceiverInputDStream(ReceiverInputDStream<scala.Tuple2<K,V>> receiverInputDStream,
                                                                               scala.reflect.ClassTag<K> evidence$1,
                                                                               scala.reflect.ClassTag<V> evidence$2)

Convert a scala ReceiverInputDStream to a Java-friendly JavaReceiverInputDStream.

Parameters:: receiverInputDStream - (undocumented); evidence$1 - (undocumented); evidence$2 - (undocumented)
Returns:: (undocumented)

scalaIntToJavaLong

public static JavaDStream<Long> scalaIntToJavaLong(DStream<Object> in)

print
```
public static void print()
```

print
```
public static void print(int num)
```

count

public static JavaDStream<Long> count()

countByValue

public static JavaPairDStream<T,Long> countByValue()

countByValue

public static JavaPairDStream<T,Long> countByValue(int numPartitions)

countByWindow

public static JavaDStream<Long> countByWindow(Duration windowDuration,
                                              Duration slideDuration)

countByValueAndWindow

public static JavaPairDStream<T,Long> countByValueAndWindow(Duration windowDuration,
                                                            Duration slideDuration)

countByValueAndWindow

public static JavaPairDStream<T,Long> countByValueAndWindow(Duration windowDuration,
                                                            Duration slideDuration,
                                                            int numPartitions)

glom

public static JavaDStream<java.util.List<T>> glom()

context

public static StreamingContext context()

map

public static <U> JavaDStream<U> map(Function<T,U> f)

mapToPair

public static <K2,V2> JavaPairDStream<K2,V2> mapToPair(PairFunction<T,K2,V2> f)

flatMap

public static <U> JavaDStream<U> flatMap(FlatMapFunction<T,U> f)

flatMapToPair

public static <K2,V2> JavaPairDStream<K2,V2> flatMapToPair(PairFlatMapFunction<T,K2,V2> f)

mapPartitions

public static <U> JavaDStream<U> mapPartitions(FlatMapFunction<java.util.Iterator<T>,U> f)

mapPartitionsToPair

public static <K2,V2> JavaPairDStream<K2,V2> mapPartitionsToPair(PairFlatMapFunction<java.util.Iterator<T>,K2,V2> f)

reduce

public static JavaDStream<T> reduce(Function2<T,T,T> f)

reduceByWindow

public static JavaDStream<T> reduceByWindow(Function2<T,T,T> reduceFunc,
                                            Duration windowDuration,
                                            Duration slideDuration)

reduceByWindow

public static JavaDStream<T> reduceByWindow(Function2<T,T,T> reduceFunc,
                                            Function2<T,T,T> invReduceFunc,
                                            Duration windowDuration,
                                            Duration slideDuration)

slice

public static java.util.List<R> slice(Time fromTime,
                                      Time toTime)

foreachRDD

public static void foreachRDD(VoidFunction<R> foreachFunc)

foreachRDD

public static void foreachRDD(VoidFunction2<R,Time> foreachFunc)

transform

public static <U> JavaDStream<U> transform(Function<R,JavaRDD<U>> transformFunc)

transform

public static <U> JavaDStream<U> transform(Function2<R,Time,JavaRDD<U>> transformFunc)

transformToPair

public static <K2,V2> JavaPairDStream<K2,V2> transformToPair(Function<R,JavaPairRDD<K2,V2>> transformFunc)

transformToPair

public static <K2,V2> JavaPairDStream<K2,V2> transformToPair(Function2<R,Time,JavaPairRDD<K2,V2>> transformFunc)

transformWith

public static <U,W> JavaDStream<W> transformWith(JavaDStream<U> other,
                                                 Function3<R,JavaRDD<U>,Time,JavaRDD<W>> transformFunc)

transformWithToPair

public static <U,K2,V2> JavaPairDStream<K2,V2> transformWithToPair(JavaDStream<U> other,
                                                                   Function3<R,JavaRDD<U>,Time,JavaPairRDD<K2,V2>> transformFunc)

transformWith

public static <K2,V2,W> JavaDStream<W> transformWith(JavaPairDStream<K2,V2> other,
                                                     Function3<R,JavaPairRDD<K2,V2>,Time,JavaRDD<W>> transformFunc)

transformWithToPair

public static <K2,V2,K3,V3> JavaPairDStream<K3,V3> transformWithToPair(JavaPairDStream<K2,V2> other,
                                                                       Function3<R,JavaPairRDD<K2,V2>,Time,JavaPairRDD<K3,V3>> transformFunc)

checkpoint

public static DStream<T> checkpoint(Duration interval)

dstream

public static DStream<scala.Tuple2<K,V>> dstream()

kManifest

public static scala.reflect.ClassTag<K> kManifest()

vManifest

public static scala.reflect.ClassTag<V> vManifest()

wrapRDD

public static JavaPairRDD<K,V> wrapRDD(RDD<scala.Tuple2<K,V>> rdd)

filter

public static JavaPairDStream<K,V> filter(Function<scala.Tuple2<K,V>,Boolean> f)

cache

public static JavaPairDStream<K,V> cache()

persist

public static JavaPairDStream<K,V> persist()

persist

public static JavaPairDStream<K,V> persist(StorageLevel storageLevel)

repartition

public static JavaPairDStream<K,V> repartition(int numPartitions)

compute

public static JavaPairRDD<K,V> compute(Time validTime)

window

public static JavaPairDStream<K,V> window(Duration windowDuration)

window

public static JavaPairDStream<K,V> window(Duration windowDuration,
                                          Duration slideDuration)

union

public static JavaPairDStream<K,V> union(JavaPairDStream<K,V> that)

groupByKey

public static JavaPairDStream<K,Iterable<V>> groupByKey()

groupByKey

public static JavaPairDStream<K,Iterable<V>> groupByKey(int numPartitions)

groupByKey

public static JavaPairDStream<K,Iterable<V>> groupByKey(Partitioner partitioner)

reduceByKey

public static JavaPairDStream<K,V> reduceByKey(Function2<V,V,V> func)

reduceByKey

public static JavaPairDStream<K,V> reduceByKey(Function2<V,V,V> func,
                                               int numPartitions)

reduceByKey

public static JavaPairDStream<K,V> reduceByKey(Function2<V,V,V> func,
                                               Partitioner partitioner)

combineByKey

public static <C> JavaPairDStream<K,C> combineByKey(Function<V,C> createCombiner,
                                                    Function2<C,V,C> mergeValue,
                                                    Function2<C,C,C> mergeCombiners,
                                                    Partitioner partitioner)

combineByKey

public static <C> JavaPairDStream<K,C> combineByKey(Function<V,C> createCombiner,
                                                    Function2<C,V,C> mergeValue,
                                                    Function2<C,C,C> mergeCombiners,
                                                    Partitioner partitioner,
                                                    boolean mapSideCombine)

groupByKeyAndWindow

public static JavaPairDStream<K,Iterable<V>> groupByKeyAndWindow(Duration windowDuration)

groupByKeyAndWindow

public static JavaPairDStream<K,Iterable<V>> groupByKeyAndWindow(Duration windowDuration,
                                                                 Duration slideDuration)

groupByKeyAndWindow

public static JavaPairDStream<K,Iterable<V>> groupByKeyAndWindow(Duration windowDuration,
                                                                 Duration slideDuration,
                                                                 int numPartitions)

groupByKeyAndWindow

public static JavaPairDStream<K,Iterable<V>> groupByKeyAndWindow(Duration windowDuration,
                                                                 Duration slideDuration,
                                                                 Partitioner partitioner)

reduceByKeyAndWindow

public static JavaPairDStream<K,V> reduceByKeyAndWindow(Function2<V,V,V> reduceFunc,
                                                        Duration windowDuration)

reduceByKeyAndWindow

public static JavaPairDStream<K,V> reduceByKeyAndWindow(Function2<V,V,V> reduceFunc,
                                                        Duration windowDuration,
                                                        Duration slideDuration)

reduceByKeyAndWindow

public static JavaPairDStream<K,V> reduceByKeyAndWindow(Function2<V,V,V> reduceFunc,
                                                        Duration windowDuration,
                                                        Duration slideDuration,
                                                        int numPartitions)

reduceByKeyAndWindow

public static JavaPairDStream<K,V> reduceByKeyAndWindow(Function2<V,V,V> reduceFunc,
                                                        Duration windowDuration,
                                                        Duration slideDuration,
                                                        Partitioner partitioner)

reduceByKeyAndWindow

public static JavaPairDStream<K,V> reduceByKeyAndWindow(Function2<V,V,V> reduceFunc,
                                                        Function2<V,V,V> invReduceFunc,
                                                        Duration windowDuration,
                                                        Duration slideDuration)

reduceByKeyAndWindow

public static JavaPairDStream<K,V> reduceByKeyAndWindow(Function2<V,V,V> reduceFunc,
                                                        Function2<V,V,V> invReduceFunc,
                                                        Duration windowDuration,
                                                        Duration slideDuration,
                                                        int numPartitions,
                                                        Function<scala.Tuple2<K,V>,Boolean> filterFunc)

reduceByKeyAndWindow

public static JavaPairDStream<K,V> reduceByKeyAndWindow(Function2<V,V,V> reduceFunc,
                                                        Function2<V,V,V> invReduceFunc,
                                                        Duration windowDuration,
                                                        Duration slideDuration,
                                                        Partitioner partitioner,
                                                        Function<scala.Tuple2<K,V>,Boolean> filterFunc)

mapWithState

public static <StateType,MappedType> JavaMapWithStateDStream<K,V,StateType,MappedType> mapWithState(StateSpec<K,V,StateType,MappedType> spec)

updateStateByKey

public static <S> JavaPairDStream<K,S> updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc)

updateStateByKey

public static <S> JavaPairDStream<K,S> updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc,
                                                        int numPartitions)

updateStateByKey

public static <S> JavaPairDStream<K,S> updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc,
                                                        Partitioner partitioner)

updateStateByKey

public static <S> JavaPairDStream<K,S> updateStateByKey(Function2<java.util.List<V>,Optional<S>,Optional<S>> updateFunc,
                                                        Partitioner partitioner,
                                                        JavaPairRDD<K,S> initialRDD)

mapValues

public static <U> JavaPairDStream<K,U> mapValues(Function<V,U> f)

flatMapValues

public static <U> JavaPairDStream<K,U> flatMapValues(Function<V,Iterable<U>> f)

cogroup

public static <W> JavaPairDStream<K,scala.Tuple2<Iterable<V>,Iterable<W>>> cogroup(JavaPairDStream<K,W> other)

cogroup

public static <W> JavaPairDStream<K,scala.Tuple2<Iterable<V>,Iterable<W>>> cogroup(JavaPairDStream<K,W> other,
                                                                                   int numPartitions)

cogroup

public static <W> JavaPairDStream<K,scala.Tuple2<Iterable<V>,Iterable<W>>> cogroup(JavaPairDStream<K,W> other,
                                                                                   Partitioner partitioner)

join

public static <W> JavaPairDStream<K,scala.Tuple2<V,W>> join(JavaPairDStream<K,W> other)

join

public static <W> JavaPairDStream<K,scala.Tuple2<V,W>> join(JavaPairDStream<K,W> other,
                                                            int numPartitions)

join

public static <W> JavaPairDStream<K,scala.Tuple2<V,W>> join(JavaPairDStream<K,W> other,
                                                            Partitioner partitioner)

leftOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<V,Optional<W>>> leftOuterJoin(JavaPairDStream<K,W> other)

leftOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<V,Optional<W>>> leftOuterJoin(JavaPairDStream<K,W> other,
                                                                               int numPartitions)

leftOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<V,Optional<W>>> leftOuterJoin(JavaPairDStream<K,W> other,
                                                                               Partitioner partitioner)

rightOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,W>> rightOuterJoin(JavaPairDStream<K,W> other)

rightOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,W>> rightOuterJoin(JavaPairDStream<K,W> other,
                                                                                int numPartitions)

rightOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,W>> rightOuterJoin(JavaPairDStream<K,W> other,
                                                                                Partitioner partitioner)

fullOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,Optional<W>>> fullOuterJoin(JavaPairDStream<K,W> other)

fullOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,Optional<W>>> fullOuterJoin(JavaPairDStream<K,W> other,
                                                                                         int numPartitions)

fullOuterJoin

public static <W> JavaPairDStream<K,scala.Tuple2<Optional<V>,Optional<W>>> fullOuterJoin(JavaPairDStream<K,W> other,
                                                                                         Partitioner partitioner)

saveAsHadoopFiles

public static void saveAsHadoopFiles(String prefix,
                                     String suffix)

saveAsHadoopFiles

public static <F extends org.apache.hadoop.mapred.OutputFormat<?,?>> void saveAsHadoopFiles(String prefix,
                                                                                            String suffix,
                                                                                            Class<?> keyClass,
                                                                                            Class<?> valueClass,
                                                                                            Class<F> outputFormatClass)

saveAsHadoopFiles

public static <F extends org.apache.hadoop.mapred.OutputFormat<?,?>> void saveAsHadoopFiles(String prefix,
                                                                                            String suffix,
                                                                                            Class<?> keyClass,
                                                                                            Class<?> valueClass,
                                                                                            Class<F> outputFormatClass,
                                                                                            org.apache.hadoop.mapred.JobConf conf)

saveAsNewAPIHadoopFiles

public static void saveAsNewAPIHadoopFiles(String prefix,
                                           String suffix)

saveAsNewAPIHadoopFiles

public static <F extends org.apache.hadoop.mapreduce.OutputFormat<?,?>> void saveAsNewAPIHadoopFiles(String prefix,
                                                                                                     String suffix,
                                                                                                     Class<?> keyClass,
                                                                                                     Class<?> valueClass,
                                                                                                     Class<F> outputFormatClass)

saveAsNewAPIHadoopFiles

public static <F extends org.apache.hadoop.mapreduce.OutputFormat<?,?>> void saveAsNewAPIHadoopFiles(String prefix,
                                                                                                     String suffix,
                                                                                                     Class<?> keyClass,
                                                                                                     Class<?> valueClass,
                                                                                                     Class<F> outputFormatClass,
                                                                                                     org.apache.hadoop.conf.Configuration conf)

toJavaDStream

public static JavaDStream<scala.Tuple2<K,V>> toJavaDStream()

classTag

public static scala.reflect.ClassTag<scala.Tuple2<K,V>> classTag()

saveAsNewAPIHadoopFiles$default$6

public static <F extends org.apache.hadoop.mapreduce.OutputFormat<?,?>> org.apache.hadoop.conf.Configuration saveAsNewAPIHadoopFiles$default$6()

inputDStream

public static InputDStream<scala.Tuple2<K,V>> inputDStream()

receiverInputDStream

public ReceiverInputDStream<scala.Tuple2<K,V>> receiverInputDStream()

kClassTag
```
public scala.reflect.ClassTag<K> kClassTag()
```
Overrides:

kClassTag in class JavaPairInputDStream<K,V>

vClassTag
```
public scala.reflect.ClassTag<V> vClassTag()
```
Overrides:

vClassTag in class JavaPairInputDStream<K,V>

Class JavaPairReceiverInputDStream<K,V>

Constructor Summary

Method Summary

Methods inherited from class org.apache.spark.streaming.api.java.JavaPairInputDStream

Methods inherited from class org.apache.spark.streaming.api.java.JavaPairDStream

Methods inherited from class Object

Methods inherited from interface org.apache.spark.streaming.api.java.JavaDStreamLike

Constructor Detail

JavaPairReceiverInputDStream

Method Detail

fromReceiverInputDStream

scalaIntToJavaLong

print

print

count

countByValue

countByValue

countByWindow

countByValueAndWindow

countByValueAndWindow

glom

context

map

mapToPair

flatMap

flatMapToPair

mapPartitions

mapPartitionsToPair

reduce

reduceByWindow

reduceByWindow

slice

foreachRDD

foreachRDD

transform

transform

transformToPair

transformToPair

transformWith

transformWithToPair

transformWith

transformWithToPair

checkpoint

dstream

kManifest

vManifest

wrapRDD

filter

cache

persist

persist

repartition

compute

window

window

union

groupByKey

groupByKey

groupByKey

reduceByKey

reduceByKey

reduceByKey

combineByKey

combineByKey

groupByKeyAndWindow

groupByKeyAndWindow

groupByKeyAndWindow

groupByKeyAndWindow

reduceByKeyAndWindow

reduceByKeyAndWindow

reduceByKeyAndWindow

reduceByKeyAndWindow

reduceByKeyAndWindow

reduceByKeyAndWindow

reduceByKeyAndWindow

mapWithState

updateStateByKey

updateStateByKey

updateStateByKey

updateStateByKey