org.apache.spark.ui.storage

Class StorageListener

Object

org.apache.spark.ui.storage.StorageListener

All Implemented Interfaces:

SparkListener

public class StorageListener
extends Object
implements SparkListener

:: DeveloperApi :: A SparkListener that prepares information to be displayed on the BlockManagerUI.

Constructor Summary

Constructors

Constructor and Description
StorageListener(StorageStatusListener storageStatusListener)

Method Summary

Methods

Modifier and Type	Method and Description
void	onStageCompleted(SparkListenerStageCompleted stageCompleted) Called when a stage completes successfully or fails, with information on the completed stage.
void	onStageSubmitted(SparkListenerStageSubmitted stageSubmitted) Called when a stage is submitted
void	onTaskEnd(SparkListenerTaskEnd taskEnd) Assumes the storage status list is fully up-to-date.
void	onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD) Called when an RDD is manually unpersisted by the application
scala.collection.Seq<RDDInfo>	rddInfoList() Filter RDD info to include only those with cached partitions
scala.collection.Seq<StorageStatus>	storageStatusList()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.scheduler.SparkListener

onApplicationEnd, onApplicationStart, onBlockManagerAdded, onBlockManagerRemoved, onEnvironmentUpdate, onJobEnd, onJobStart, onTaskGettingResult, onTaskStart

Constructor Detail

StorageListener

public StorageListener(StorageStatusListener storageStatusListener)

Method Detail

storageStatusList

public scala.collection.Seq<StorageStatus> storageStatusList()

rddInfoList

public scala.collection.Seq<RDDInfo> rddInfoList()

Filter RDD info to include only those with cached partitions

onTaskEnd

public void onTaskEnd(SparkListenerTaskEnd taskEnd)

Assumes the storage status list is fully up-to-date. This implies the corresponding StorageStatusSparkListener must process the SparkListenerTaskEnd event before this listener.

Specified by:

onTaskEnd in interface SparkListener

onStageSubmitted

public void onStageSubmitted(SparkListenerStageSubmitted stageSubmitted)

Description copied from interface: SparkListener

Called when a stage is submitted

Specified by:

onStageSubmitted in interface SparkListener

onStageCompleted

public void onStageCompleted(SparkListenerStageCompleted stageCompleted)

Description copied from interface: SparkListener

Called when a stage completes successfully or fails, with information on the completed stage.

Specified by:

onStageCompleted in interface SparkListener

onUnpersistRDD

public void onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD)

Description copied from interface: SparkListener

Called when an RDD is manually unpersisted by the application

Specified by:

onUnpersistRDD in interface SparkListener