org.apache.spark.scheduler
Interface SparkListener

All Known Implementing Classes:

EnvironmentListener, ExecutorsListener, JavaSparkListener, JobLogger, JobProgressListener, SparkFirehoseListener, StatsReportListener, StorageListener, StorageStatusListener

public interface SparkListener

:: DeveloperApi :: Interface for listening to events from the Spark scheduler. Note that this is an internal interface which might change in different Spark releases. Java clients should extend JavaSparkListener

Method Summary

void	onApplicationEnd(SparkListenerApplicationEnd applicationEnd) Called when the application ends
void	onApplicationStart(SparkListenerApplicationStart applicationStart) Called when the application starts
void	onBlockManagerAdded(SparkListenerBlockManagerAdded blockManagerAdded) Called when a new block manager has joined
void	onBlockManagerRemoved(SparkListenerBlockManagerRemoved blockManagerRemoved) Called when an existing block manager has been removed
void	onEnvironmentUpdate(SparkListenerEnvironmentUpdate environmentUpdate) Called when environment properties have been updated
void	onExecutorAdded(SparkListenerExecutorAdded executorAdded) Called when the driver registers a new executor.
void	onExecutorMetricsUpdate(SparkListenerExecutorMetricsUpdate executorMetricsUpdate) Called when the driver receives task metrics from an executor in a heartbeat.
void	onExecutorRemoved(SparkListenerExecutorRemoved executorRemoved) Called when the driver removes an executor.
void	onJobEnd(SparkListenerJobEnd jobEnd) Called when a job ends
void	onJobStart(SparkListenerJobStart jobStart) Called when a job starts
void	onStageCompleted(SparkListenerStageCompleted stageCompleted) Called when a stage completes successfully or fails, with information on the completed stage.
void	onStageSubmitted(SparkListenerStageSubmitted stageSubmitted) Called when a stage is submitted
void	onTaskEnd(SparkListenerTaskEnd taskEnd) Called when a task ends
void	onTaskGettingResult(SparkListenerTaskGettingResult taskGettingResult) Called when a task begins remotely fetching its result (will not be called for tasks that do not need to fetch the result remotely).
void	onTaskStart(SparkListenerTaskStart taskStart) Called when a task starts
void	onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD) Called when an RDD is manually unpersisted by the application

Method Detail

onStageCompleted

void onStageCompleted(SparkListenerStageCompleted stageCompleted)

Called when a stage completes successfully or fails, with information on the completed stage.

Parameters:

stageCompleted - (undocumented)

onStageSubmitted

void onStageSubmitted(SparkListenerStageSubmitted stageSubmitted)

Called when a stage is submitted

Parameters:

stageSubmitted - (undocumented)

onTaskStart

void onTaskStart(SparkListenerTaskStart taskStart)

Called when a task starts

Parameters:

taskStart - (undocumented)

onTaskGettingResult

void onTaskGettingResult(SparkListenerTaskGettingResult taskGettingResult)

Called when a task begins remotely fetching its result (will not be called for tasks that do not need to fetch the result remotely).

Parameters:

taskGettingResult - (undocumented)

onTaskEnd

void onTaskEnd(SparkListenerTaskEnd taskEnd)

Called when a task ends

Parameters:

taskEnd - (undocumented)

onJobStart

void onJobStart(SparkListenerJobStart jobStart)

Called when a job starts

Parameters:

jobStart - (undocumented)

onJobEnd

void onJobEnd(SparkListenerJobEnd jobEnd)

Called when a job ends

Parameters:

jobEnd - (undocumented)

onEnvironmentUpdate

void onEnvironmentUpdate(SparkListenerEnvironmentUpdate environmentUpdate)

Called when environment properties have been updated

Parameters:

environmentUpdate - (undocumented)

onBlockManagerAdded

void onBlockManagerAdded(SparkListenerBlockManagerAdded blockManagerAdded)

Called when a new block manager has joined

Parameters:

blockManagerAdded - (undocumented)

onBlockManagerRemoved

void onBlockManagerRemoved(SparkListenerBlockManagerRemoved blockManagerRemoved)

Called when an existing block manager has been removed

Parameters:

blockManagerRemoved - (undocumented)

onUnpersistRDD

void onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD)

Called when an RDD is manually unpersisted by the application

Parameters:

unpersistRDD - (undocumented)

onApplicationStart

void onApplicationStart(SparkListenerApplicationStart applicationStart)

Called when the application starts

Parameters:

applicationStart - (undocumented)

onApplicationEnd

void onApplicationEnd(SparkListenerApplicationEnd applicationEnd)

Called when the application ends

Parameters:

applicationEnd - (undocumented)

onExecutorMetricsUpdate

void onExecutorMetricsUpdate(SparkListenerExecutorMetricsUpdate executorMetricsUpdate)

Called when the driver receives task metrics from an executor in a heartbeat.

Parameters:

executorMetricsUpdate - (undocumented)

onExecutorAdded

void onExecutorAdded(SparkListenerExecutorAdded executorAdded)

Called when the driver registers a new executor.

Parameters:

executorAdded - (undocumented)

onExecutorRemoved

void onExecutorRemoved(SparkListenerExecutorRemoved executorRemoved)

Called when the driver removes an executor.

Parameters:

executorRemoved - (undocumented)