org.apache.spark.api.r

Class RRDD<T>

java.lang.Object

org.apache.spark.rdd.RDD<U>

org.apache.spark.api.r.BaseRRDD<T,byte[]>

org.apache.spark.api.r.RRDD<T>

All Implemented Interfaces:

java.io.Serializable, Logging

public class RRDD<T>
extends BaseRRDD<T,byte[]>

An RDD that stores serialized R objects as Array[Byte].

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
RRDD(RDD<T> parent, byte[] func, java.lang.String deserializer, java.lang.String serializer, byte[] packageNames, java.lang.Object[] broadcastVars, scala.reflect.ClassTag<T> evidence$4)

Method Summary

Methods

Modifier and Type	Method and Description
JavaRDD<byte[]>	asJavaRDD()
static JavaRDD<byte[]>	createRDDFromArray(JavaSparkContext jsc, byte[][] arr) Create an RRDD given a sequence of byte arrays.
static org.apache.spark.api.r.BufferedStreamThread	createRWorker(int port) ProcessBuilder used to launch worker R processes.
static JavaSparkContext	createSparkContext(java.lang.String master, java.lang.String appName, java.lang.String sparkHome, java.lang.String[] jars, java.util.Map<java.lang.Object,java.lang.Object> sparkEnvirMap, java.util.Map<java.lang.Object,java.lang.Object> sparkExecutorEnvMap)
protected byte[]	readData(int length)

Methods inherited from class org.apache.spark.api.r.BaseRRDD

compute, dataStream, getPartitions, read

Methods inherited from class org.apache.spark.rdd.RDD

aggregate, cache, cartesian, checkpoint, checkpointData, clearDependencies, coalesce, collect, collect, context, count, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, creationSite, dependencies, distinct, distinct, doubleRDDToDoubleRDDFunctions, filter, filterWith, first, firstParent, flatMap, flatMapWith, fold, foreach, foreachPartition, foreachWith, getCheckpointFile, getDependencies, getPreferredLocations, getStorageLevel, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isCheckpointed, isEmpty, iterator, keyBy, localCheckpoint, map, mapPartitions, mapPartitionsWithContext, mapPartitionsWithIndex, mapPartitionsWithSplit, mapWith, max, min, name, numericRDDToDoubleRDDFunctions, parent, partitioner, partitions, persist, persist, pipe, pipe, pipe, preferredLocations, randomSplit, rddToAsyncRDDActions, rddToOrderedRDDFunctions, rddToPairRDDFunctions, rddToSequenceFileRDDFunctions, reduce, repartition, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, scope, setName, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toArray, toDebugString, toJavaRDD, toLocalIterator, top, toString, treeAggregate, treeReduce, union, unpersist, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipWithIndex, zipWithUniqueId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

RRDD

public RRDD(RDD<T> parent,
    byte[] func,
    java.lang.String deserializer,
    java.lang.String serializer,
    byte[] packageNames,
    java.lang.Object[] broadcastVars,
    scala.reflect.ClassTag<T> evidence$4)

Method Detail

createSparkContext

public static JavaSparkContext createSparkContext(java.lang.String master,
                                  java.lang.String appName,
                                  java.lang.String sparkHome,
                                  java.lang.String[] jars,
                                  java.util.Map<java.lang.Object,java.lang.Object> sparkEnvirMap,
                                  java.util.Map<java.lang.Object,java.lang.Object> sparkExecutorEnvMap)

createRWorker

public static org.apache.spark.api.r.BufferedStreamThread createRWorker(int port)

ProcessBuilder used to launch worker R processes.

Parameters:

port - (undocumented)

Returns:

(undocumented)

createRDDFromArray

public static JavaRDD<byte[]> createRDDFromArray(JavaSparkContext jsc,
                                 byte[][] arr)

Create an RRDD given a sequence of byte arrays. Used to create RRDD when parallelize is called from R.

Parameters:

jsc - (undocumented)

arr - (undocumented)

Returns:

(undocumented)

readData

protected byte[] readData(int length)

Specified by:

readData in class BaseRRDD<T,byte[]>

asJavaRDD

public JavaRDD<byte[]> asJavaRDD()