Package com.linkedin.venice.partitioner

Class VenicePartitioner

java.lang.Object

com.linkedin.venice.partitioner.VenicePartitioner

Direct Known Subclasses:

ConstantVenicePartitioner, DefaultVenicePartitioner, InvalidKeySchemaPartitioner, SimplePartitioner, UserPartitionAwarePartitioner

public abstract class VenicePartitioner
extends java.lang.Object

Determines partitioning, which is used for producing messages into the right Kafka partitions and routing reads to the correct Venice storage nodes. N.B.: This is purposefully de-coupled from Kafka, so that the Router does not need to depend on Kafka.

Field Summary

Fields

Modifier and Type	Field	Description
protected VeniceProperties	props

Constructor Summary

Constructors

Constructor	Description
VenicePartitioner()
VenicePartitioner(VeniceProperties props)
VenicePartitioner(VeniceProperties props, org.apache.avro.Schema schema)

Method Summary

Modifier and Type	Method	Description
protected void	checkSchema(org.apache.avro.Schema keySchema)	Implementors of this class can optionally provide an implementation of this function, which can perform validation of schemas to be certain that they are compatible with the partitioner implementation.
abstract int	getPartitionId(byte[] keyBytes, int numPartitions)	A function that returns the partitionId based on the key and partition count.
int	getPartitionId(byte[] keyBytes, int offset, int length, int numPartitions)	Implementors of this class can optionally provide an implementation of this function, which would result in eliminating an instantiation of ByteBuffer in the case where the provided offset and length do not map to the boundaries of the byte[].
abstract int	getPartitionId(java.nio.ByteBuffer keyByteBuffer, int numPartitions)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

props

protected final VeniceProperties props

Constructor Detail

VenicePartitioner

public VenicePartitioner()

VenicePartitioner

public VenicePartitioner(VeniceProperties props)

VenicePartitioner

public VenicePartitioner(VeniceProperties props,
                         org.apache.avro.Schema schema)

Method Detail

getPartitionId

public abstract int getPartitionId(byte[] keyBytes,
                                   int numPartitions)

A function that returns the partitionId based on the key and partition count.

Parameters:

keyBytes - - A key that will be mapped into a partition

numPartitions - - The number of total partitions available in Kafka/storage

Returns:

getPartitionId

public int getPartitionId(byte[] keyBytes,
                          int offset,
                          int length,
                          int numPartitions)

Implementors of this class can optionally provide an implementation of this function, which would result in eliminating an instantiation of ByteBuffer in the case where the provided offset and length do not map to the boundaries of the byte[]. This is just a minor optimization.

getPartitionId

public abstract int getPartitionId(java.nio.ByteBuffer keyByteBuffer,
                                   int numPartitions)

checkSchema

protected void checkSchema(@Nonnull
                           org.apache.avro.Schema keySchema)
                    throws PartitionerSchemaMismatchException

Implementors of this class can optionally provide an implementation of this function, which can perform validation of schemas to be certain that they are compatible with the partitioner implementation.

Parameters:

keySchema - the schema to be validated

Throws:

PartitionerSchemaMismatchException - should the provided schema not match the partitioner (with a message that explains why).