org.apache.spark.streaming.scheduler

Class StatsReportListener

Object

org.apache.spark.streaming.scheduler.StatsReportListener

All Implemented Interfaces:

StreamingListener

public class StatsReportListener
extends Object
implements StreamingListener

:: DeveloperApi :: A simple StreamingListener that logs summary statistics across Spark Streaming batches param: numBatchInfos Number of last batches to consider for generating statistics (default: 10)

Constructor Summary

Constructors

Constructor and Description
StatsReportListener(int numBatchInfos)

Method Summary

Modifier and Type	Method and Description
scala.collection.mutable.Queue<BatchInfo>	batchInfos()
scala.Option<org.apache.spark.util.Distribution>	extractDistribution(scala.Function1<BatchInfo,scala.Option<Object>> getMetric)
void	onBatchCompleted(StreamingListenerBatchCompleted batchStarted) Called when processing of a batch of jobs has completed.
void	printStats()
void	showMillisDistribution(String heading, scala.Function1<BatchInfo,scala.Option<Object>> getMetric)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.streaming.scheduler.StreamingListener

onBatchStarted, onBatchSubmitted, onOutputOperationCompleted, onOutputOperationStarted, onReceiverError, onReceiverStarted, onReceiverStopped, onStreamingStarted

Constructor Detail

StatsReportListener

public StatsReportListener(int numBatchInfos)

Method Detail

batchInfos

public scala.collection.mutable.Queue<BatchInfo> batchInfos()

onBatchCompleted

public void onBatchCompleted(StreamingListenerBatchCompleted batchStarted)

Description copied from interface: StreamingListener

Called when processing of a batch of jobs has completed.

Specified by:

onBatchCompleted in interface StreamingListener

printStats

public void printStats()

showMillisDistribution

public void showMillisDistribution(String heading,
                                   scala.Function1<BatchInfo,scala.Option<Object>> getMetric)

extractDistribution

public scala.Option<org.apache.spark.util.Distribution> extractDistribution(scala.Function1<BatchInfo,scala.Option<Object>> getMetric)