org.apache.spark.input

Class StreamBasedRecordReader<T>

Object

org.apache.hadoop.mapreduce.RecordReader<String,T>

org.apache.spark.input.StreamBasedRecordReader<T>

All Implemented Interfaces:

java.io.Closeable, AutoCloseable

Direct Known Subclasses:

StreamRecordReader

public abstract class StreamBasedRecordReader<T>
extends org.apache.hadoop.mapreduce.RecordReader<String,T>

An abstract class of RecordReader to reading files out as streams

Constructor Summary

Constructors

Constructor and Description
StreamBasedRecordReader(org.apache.hadoop.mapreduce.lib.input.CombineFileSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context, Integer index)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
String	getCurrentKey()
T	getCurrentValue()
float	getProgress()
void	initialize(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
boolean	nextKeyValue()
abstract T	parseStream(PortableDataStream inStream) Parse the stream (and close it afterwards) and return the value as in type T

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamBasedRecordReader

public StreamBasedRecordReader(org.apache.hadoop.mapreduce.lib.input.CombineFileSplit split,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context,
                       Integer index)

Method Detail

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit split,
              org.apache.hadoop.mapreduce.TaskAttemptContext context)

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<String,T>

close

public void close()

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<String,T>

getProgress

public float getProgress()

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<String,T>

getCurrentKey

public String getCurrentKey()

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<String,T>

getCurrentValue

public T getCurrentValue()

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<String,T>

nextKeyValue

public boolean nextKeyValue()

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<String,T>

parseStream

public abstract T parseStream(PortableDataStream inStream)

Parse the stream (and close it afterwards) and return the value as in type T

Parameters:

inStream - the stream to be read in

Returns:

the data formatted as