Package com.linkedin.venice.spark.datawriter.partition

Class VeniceSparkPartitioner

java.lang.Object

org.apache.spark.Partitioner

com.linkedin.venice.spark.datawriter.partition.VeniceSparkPartitioner

All Implemented Interfaces:

Serializable, scala.Serializable

public class VeniceSparkPartitioner
extends org.apache.spark.Partitioner

VeniceSparkPartitioner will use the exact same partitioner as the one that will be used inside VeniceWriter (e.g. DefaultVenicePartitioner) to make sure the messages inside each partition belong to the same PubSub topic partition.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
VeniceSparkPartitioner(org.apache.spark.broadcast.Broadcast<Properties> broadcastProperties, int numPartitions)

Method Summary

Modifier and Type	Method	Description
int	getPartition(Object key)
int	numPartitions()

Methods inherited from class org.apache.spark.Partitioner

defaultPartitioner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VeniceSparkPartitioner

public VeniceSparkPartitioner(org.apache.spark.broadcast.Broadcast<Properties> broadcastProperties,
 int numPartitions)

Method Details

numPartitions

public int numPartitions()

Specified by:

numPartitions in class org.apache.spark.Partitioner

getPartition

public int getPartition(Object key)

Specified by:

getPartition in class org.apache.spark.Partitioner