Package com.linkedin.venice.hadoop.spark.datawriter.partition

Class VeniceSparkPartitioner

  • All Implemented Interfaces:
    java.io.Serializable, scala.Serializable
    public class VeniceSparkPartitioner
extends org.apache.spark.Partitioner
    VeniceSparkPartitioner will use the exact same partitioner as the one that will be used inside VeniceWriter (e.g. DefaultVenicePartitioner) to make sure the messages inside each partition belong to the same PubSub topic partition.
    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      VeniceSparkPartitioner​(org.apache.spark.broadcast.Broadcast<java.util.Properties> broadcastProperties, int numPartitions)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      int getPartition​(java.lang.Object key)  
      int numPartitions()  

      • Methods inherited from class org.apache.spark.Partitioner

        defaultPartitioner

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • VeniceSparkPartitioner

        public VeniceSparkPartitioner​(org.apache.spark.broadcast.Broadcast<java.util.Properties> broadcastProperties,
                              int numPartitions)

    • Method Detail

      • numPartitions

        public int numPartitions()
        Specified by:
        numPartitions in class org.apache.spark.Partitioner

      • getPartition

        public int getPartition​(java.lang.Object key)
        Specified by:
        getPartition in class org.apache.spark.Partitioner