org.apache.hadoop.hive.ql.io.orc

Class SparkOrcNewRecordReader

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

org.apache.hadoop.hive.ql.io.orc.SparkOrcNewRecordReader

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public class SparkOrcNewRecordReader
extends org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

This is based on hive-exec-1.2.1 OrcNewInputFormat.OrcRecordReader. This class exposes getObjectInspector which can be used for reducing NameNode calls in OrcRelation.

Constructor Summary

Constructors

Constructor and Description
SparkOrcNewRecordReader(org.apache.hadoop.hive.ql.io.orc.Reader file, org.apache.hadoop.conf.Configuration conf, long offset, long length)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
org.apache.hadoop.io.NullWritable	getCurrentKey()
org.apache.hadoop.hive.ql.io.orc.OrcStruct	getCurrentValue()
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector	getObjectInspector()
float	getProgress()
void	initialize(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
boolean	nextKeyValue()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkOrcNewRecordReader

public SparkOrcNewRecordReader(org.apache.hadoop.hive.ql.io.orc.Reader file,
                       org.apache.hadoop.conf.Configuration conf,
                       long offset,
                       long length)
                        throws java.io.IOException

Throws:

java.io.IOException

Method Detail

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

Throws:

java.io.IOException

getCurrentKey

public org.apache.hadoop.io.NullWritable getCurrentKey()
                                                throws java.io.IOException,
                                                       java.lang.InterruptedException

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

Throws:

java.io.IOException

java.lang.InterruptedException

getCurrentValue

public org.apache.hadoop.hive.ql.io.orc.OrcStruct getCurrentValue()
                                                           throws java.io.IOException,
                                                                  java.lang.InterruptedException

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

Throws:

java.io.IOException

java.lang.InterruptedException

getProgress

public float getProgress()
                  throws java.io.IOException,
                         java.lang.InterruptedException

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

Throws:

java.io.IOException

java.lang.InterruptedException

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit split,
              org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws java.io.IOException,
                       java.lang.InterruptedException

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

Throws:

java.io.IOException

java.lang.InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws java.io.IOException,
                            java.lang.InterruptedException

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.hive.ql.io.orc.OrcStruct>

Throws:

java.io.IOException

java.lang.InterruptedException

getObjectInspector

public org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector getObjectInspector()