org.apache.spark

Class SparkFirehoseListener

Object

org.apache.spark.SparkFirehoseListener

All Implemented Interfaces:

SparkListener

public class SparkFirehoseListener
extends Object
implements SparkListener

Class that allows users to receive all SparkListener events. Users should override the onEvent method. This is a concrete Java class in order to ensure that we don't forget to update it when adding new methods to SparkListener: forgetting to add a method will result in a compilation error (if this was a concrete Scala class, default implementations of new event handlers would be inherited from the SparkListener trait).

Constructor Summary

Constructors

Constructor and Description
SparkFirehoseListener()

Method Summary

Methods

Modifier and Type	Method and Description
void	onApplicationEnd(SparkListenerApplicationEnd applicationEnd) Called when the application ends
void	onApplicationStart(SparkListenerApplicationStart applicationStart) Called when the application starts
void	onBlockManagerAdded(SparkListenerBlockManagerAdded blockManagerAdded) Called when a new block manager has joined
void	onBlockManagerRemoved(SparkListenerBlockManagerRemoved blockManagerRemoved) Called when an existing block manager has been removed
void	onBlockUpdated(SparkListenerBlockUpdated blockUpdated) Called when the driver receives a block update info.
void	onEnvironmentUpdate(SparkListenerEnvironmentUpdate environmentUpdate) Called when environment properties have been updated
void	onEvent(SparkListenerEvent event)
void	onExecutorAdded(SparkListenerExecutorAdded executorAdded) Called when the driver registers a new executor.
void	onExecutorMetricsUpdate(SparkListenerExecutorMetricsUpdate executorMetricsUpdate) Called when the driver receives task metrics from an executor in a heartbeat.
void	onExecutorRemoved(SparkListenerExecutorRemoved executorRemoved) Called when the driver removes an executor.
void	onJobEnd(SparkListenerJobEnd jobEnd) Called when a job ends
void	onJobStart(SparkListenerJobStart jobStart) Called when a job starts
void	onStageCompleted(SparkListenerStageCompleted stageCompleted) Called when a stage completes successfully or fails, with information on the completed stage.
void	onStageSubmitted(SparkListenerStageSubmitted stageSubmitted) Called when a stage is submitted
void	onTaskEnd(SparkListenerTaskEnd taskEnd) Called when a task ends
void	onTaskGettingResult(SparkListenerTaskGettingResult taskGettingResult) Called when a task begins remotely fetching its result (will not be called for tasks that do not need to fetch the result remotely).
void	onTaskStart(SparkListenerTaskStart taskStart) Called when a task starts
void	onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD) Called when an RDD is manually unpersisted by the application

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkFirehoseListener

public SparkFirehoseListener()

Method Detail

onEvent

public void onEvent(SparkListenerEvent event)

onStageCompleted

public final void onStageCompleted(SparkListenerStageCompleted stageCompleted)

Description copied from interface: SparkListener

Called when a stage completes successfully or fails, with information on the completed stage.

Specified by:

onStageCompleted in interface SparkListener

Parameters:

stageCompleted - (undocumented)

onStageSubmitted

public final void onStageSubmitted(SparkListenerStageSubmitted stageSubmitted)

Description copied from interface: SparkListener

Called when a stage is submitted

Specified by:

onStageSubmitted in interface SparkListener

Parameters:

stageSubmitted - (undocumented)

onTaskStart

public final void onTaskStart(SparkListenerTaskStart taskStart)

Description copied from interface: SparkListener

Called when a task starts

Specified by:

onTaskStart in interface SparkListener

Parameters:

taskStart - (undocumented)

onTaskGettingResult

public final void onTaskGettingResult(SparkListenerTaskGettingResult taskGettingResult)

Description copied from interface: SparkListener

Called when a task begins remotely fetching its result (will not be called for tasks that do not need to fetch the result remotely).

Specified by:

onTaskGettingResult in interface SparkListener

Parameters:

taskGettingResult - (undocumented)

onTaskEnd

public final void onTaskEnd(SparkListenerTaskEnd taskEnd)

Description copied from interface: SparkListener

Called when a task ends

Specified by:

onTaskEnd in interface SparkListener

Parameters:

taskEnd - (undocumented)

onJobStart

public final void onJobStart(SparkListenerJobStart jobStart)

Description copied from interface: SparkListener

Called when a job starts

Specified by:

onJobStart in interface SparkListener

Parameters:

jobStart - (undocumented)

onJobEnd

public final void onJobEnd(SparkListenerJobEnd jobEnd)

Description copied from interface: SparkListener

Called when a job ends

Specified by:

onJobEnd in interface SparkListener

Parameters:

jobEnd - (undocumented)

onEnvironmentUpdate

public final void onEnvironmentUpdate(SparkListenerEnvironmentUpdate environmentUpdate)

Description copied from interface: SparkListener

Called when environment properties have been updated

Specified by:

onEnvironmentUpdate in interface SparkListener

Parameters:

environmentUpdate - (undocumented)

onBlockManagerAdded

public final void onBlockManagerAdded(SparkListenerBlockManagerAdded blockManagerAdded)

Description copied from interface: SparkListener

Called when a new block manager has joined

Specified by:

onBlockManagerAdded in interface SparkListener

Parameters:

blockManagerAdded - (undocumented)

onBlockManagerRemoved

public final void onBlockManagerRemoved(SparkListenerBlockManagerRemoved blockManagerRemoved)

Description copied from interface: SparkListener

Called when an existing block manager has been removed

Specified by:

onBlockManagerRemoved in interface SparkListener

Parameters:

blockManagerRemoved - (undocumented)

onUnpersistRDD

public final void onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD)

Description copied from interface: SparkListener

Called when an RDD is manually unpersisted by the application

Specified by:

onUnpersistRDD in interface SparkListener

Parameters:

unpersistRDD - (undocumented)

onApplicationStart

public final void onApplicationStart(SparkListenerApplicationStart applicationStart)

Description copied from interface: SparkListener

Called when the application starts

Specified by:

onApplicationStart in interface SparkListener

Parameters:

applicationStart - (undocumented)

onApplicationEnd

public final void onApplicationEnd(SparkListenerApplicationEnd applicationEnd)

Description copied from interface: SparkListener

Called when the application ends

Specified by:

onApplicationEnd in interface SparkListener

Parameters:

applicationEnd - (undocumented)

onExecutorMetricsUpdate

public final void onExecutorMetricsUpdate(SparkListenerExecutorMetricsUpdate executorMetricsUpdate)

Description copied from interface: SparkListener

Called when the driver receives task metrics from an executor in a heartbeat.

Specified by:

onExecutorMetricsUpdate in interface SparkListener

Parameters:

executorMetricsUpdate - (undocumented)

onExecutorAdded

public final void onExecutorAdded(SparkListenerExecutorAdded executorAdded)

Description copied from interface: SparkListener

Called when the driver registers a new executor.

Specified by:

onExecutorAdded in interface SparkListener

Parameters:

executorAdded - (undocumented)

onExecutorRemoved

public final void onExecutorRemoved(SparkListenerExecutorRemoved executorRemoved)

Description copied from interface: SparkListener

Called when the driver removes an executor.

Specified by:

onExecutorRemoved in interface SparkListener

Parameters:

executorRemoved - (undocumented)

onBlockUpdated

public void onBlockUpdated(SparkListenerBlockUpdated blockUpdated)

Description copied from interface: SparkListener

Called when the driver receives a block update info.

Specified by:

onBlockUpdated in interface SparkListener

Parameters:

blockUpdated - (undocumented)