org.apache.spark.ui.storage
Class StorageListener

Object
  org.apache.spark.ui.storage.StorageListener

All Implemented Interfaces:

SparkListener

public class StorageListener
extends Object
implements SparkListener

:: DeveloperApi :: A SparkListener that prepares information to be displayed on the BlockManagerUI.

This class is thread-safe (unlike JobProgressListener)

Constructor Summary

StorageListener(StorageStatusListener storageStatusListener)

Method Summary

scala.collection.mutable.Map<Object,RDDInfo>	_rddInfoMap()
void	onStageCompleted(SparkListenerStageCompleted stageCompleted) Called when a stage completes successfully or fails, with information on the completed stage.
void	onStageSubmitted(SparkListenerStageSubmitted stageSubmitted) Called when a stage is submitted
void	onTaskEnd(SparkListenerTaskEnd taskEnd) Assumes the storage status list is fully up-to-date.
void	onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD) Called when an RDD is manually unpersisted by the application
scala.collection.Seq<RDDInfo>	rddInfoList() Filter RDD info to include only those with cached partitions
scala.collection.Seq<StorageStatus>	storageStatusList()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.scheduler.SparkListener

onApplicationEnd, onApplicationStart, onBlockManagerAdded, onBlockManagerRemoved, onEnvironmentUpdate, onExecutorAdded, onExecutorMetricsUpdate, onExecutorRemoved, onJobEnd, onJobStart, onTaskGettingResult, onTaskStart

Constructor Detail

StorageListener

public StorageListener(StorageStatusListener storageStatusListener)

Method Detail

_rddInfoMap

public scala.collection.mutable.Map<Object,RDDInfo> _rddInfoMap()

storageStatusList

public scala.collection.Seq<StorageStatus> storageStatusList()

rddInfoList

public scala.collection.Seq<RDDInfo> rddInfoList()

Filter RDD info to include only those with cached partitions

onTaskEnd

public void onTaskEnd(SparkListenerTaskEnd taskEnd)

Assumes the storage status list is fully up-to-date. This implies the corresponding StorageStatusSparkListener must process the SparkListenerTaskEnd event before this listener.

Specified by:

onTaskEnd in interface SparkListener

Parameters:

taskEnd - (undocumented)

onStageSubmitted

public void onStageSubmitted(SparkListenerStageSubmitted stageSubmitted)

Description copied from interface: SparkListener

Called when a stage is submitted

Specified by:

onStageSubmitted in interface SparkListener

Parameters:

stageSubmitted - (undocumented)

onStageCompleted

public void onStageCompleted(SparkListenerStageCompleted stageCompleted)

Description copied from interface: SparkListener

Called when a stage completes successfully or fails, with information on the completed stage.

Specified by:

onStageCompleted in interface SparkListener

Parameters:

stageCompleted - (undocumented)

onUnpersistRDD

public void onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD)

Description copied from interface: SparkListener

Called when an RDD is manually unpersisted by the application

Specified by:

onUnpersistRDD in interface SparkListener

Parameters:

unpersistRDD - (undocumented)