org.apache.spark.streaming.scheduler

Class StatsReportListener

Object

org.apache.spark.streaming.scheduler.StatsReportListener

All Implemented Interfaces:

StreamingListener

public class StatsReportListener
extends Object
implements StreamingListener

:: DeveloperApi :: A simple StreamingListener that logs summary statistics across Spark Streaming batches param: numBatchInfos Number of last batches to consider for generating statistics (default: 10)

Constructor Summary

Constructors

Constructor and Description
StatsReportListener(int numBatchInfos)

Method Summary

Modifier and Type	Method and Description
scala.collection.mutable.Queue<BatchInfo>	batchInfos()
scala.Option<org.apache.spark.util.Distribution>	extractDistribution(scala.Function1<BatchInfo,scala.Option<Object>> getMetric)
void	onBatchCompleted(StreamingListenerBatchCompleted batchStarted) Called when processing of a batch of jobs has completed.
static void	onBatchStarted(StreamingListenerBatchStarted batchStarted)
static void	onBatchSubmitted(StreamingListenerBatchSubmitted batchSubmitted)
static void	onOutputOperationCompleted(StreamingListenerOutputOperationCompleted outputOperationCompleted)
static void	onOutputOperationStarted(StreamingListenerOutputOperationStarted outputOperationStarted)
static void	onReceiverError(StreamingListenerReceiverError receiverError)
static void	onReceiverStarted(StreamingListenerReceiverStarted receiverStarted)
static void	onReceiverStopped(StreamingListenerReceiverStopped receiverStopped)
static void	onStreamingStarted(StreamingListenerStreamingStarted streamingStarted)
void	printStats()
void	showMillisDistribution(String heading, scala.Function1<BatchInfo,scala.Option<Object>> getMetric)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.streaming.scheduler.StreamingListener

onBatchStarted, onBatchSubmitted, onOutputOperationCompleted, onOutputOperationStarted, onReceiverError, onReceiverStarted, onReceiverStopped, onStreamingStarted

Constructor Detail

StatsReportListener

public StatsReportListener(int numBatchInfos)

Method Detail

onStreamingStarted

public static void onStreamingStarted(StreamingListenerStreamingStarted streamingStarted)

onReceiverStarted

public static void onReceiverStarted(StreamingListenerReceiverStarted receiverStarted)

onReceiverError

public static void onReceiverError(StreamingListenerReceiverError receiverError)

onReceiverStopped

public static void onReceiverStopped(StreamingListenerReceiverStopped receiverStopped)

onBatchSubmitted

public static void onBatchSubmitted(StreamingListenerBatchSubmitted batchSubmitted)

onBatchStarted

public static void onBatchStarted(StreamingListenerBatchStarted batchStarted)

onOutputOperationStarted

public static void onOutputOperationStarted(StreamingListenerOutputOperationStarted outputOperationStarted)

onOutputOperationCompleted

public static void onOutputOperationCompleted(StreamingListenerOutputOperationCompleted outputOperationCompleted)

batchInfos

public scala.collection.mutable.Queue<BatchInfo> batchInfos()

onBatchCompleted

public void onBatchCompleted(StreamingListenerBatchCompleted batchStarted)

Description copied from interface: StreamingListener

Called when processing of a batch of jobs has completed.

Specified by:

onBatchCompleted in interface StreamingListener

printStats

public void printStats()

showMillisDistribution

public void showMillisDistribution(String heading,
                                   scala.Function1<BatchInfo,scala.Option<Object>> getMetric)

extractDistribution

public scala.Option<org.apache.spark.util.Distribution> extractDistribution(scala.Function1<BatchInfo,scala.Option<Object>> getMetric)