Package com.linkedin.venice.vpj.pubsub.input

Class PubSubPartitionSplit

java.lang.Object

com.linkedin.venice.vpj.pubsub.input.PubSubPartitionSplit

All Implemented Interfaces:

Serializable

public class PubSubPartitionSplit
extends Object
implements Serializable

Describes a contiguous range of records within a single PubSubTopicPartition. The range is bounded by a start position (inclusive) and an end position (exclusive). It is used to enable parallel reading during repush jobs, where data is read from pub-sub topics and written into a new version of a Venice store.

Scope and constraints:

• One PubSubPartitionSplit covers exactly one topic partition.
• startPubSubPosition must be <= endPubSubPosition.
• Instances should be treated as immutable once constructed.

Engine independence: This is a framework-agnostic descriptor. Adapters can translate it to a Hadoop InputSplit or a Spark DataSource V2 InputPartition.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
PubSubPartitionSplit()	Nullary constructor for deserialization.
PubSubPartitionSplit(PubSubTopicRepository topicRepository, PubSubTopicPartition pubSubTopicPartition, PubSubPosition startPubSubPosition, PubSubPosition endPubSubPosition, long numberOfRecords, int splitIndex, long startIndex)

Method Summary

Modifier and Type	Method	Description
boolean	equals(Object o)
PubSubPosition	getEndPubSubPosition()
long	getNumberOfRecords()
int	getPartitionNumber()
PubSubTopicPartition	getPubSubTopicPartition()
int	getSplitIndex()
long	getStartIndex()
PubSubPosition	getStartPubSubPosition()
String	getTopicName()
PubSubTopicRepository	getTopicRepository()
int	hashCode()
void	readFields(DataInput in)
static PubSubPartitionSplit	readFrom(DataInput in)
String	toString()
void	write(DataOutput out)
void	writeTo(DataOutput out)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

PubSubPartitionSplit

public PubSubPartitionSplit()

Nullary constructor for deserialization.

PubSubPartitionSplit

public PubSubPartitionSplit(PubSubTopicRepository topicRepository,
 PubSubTopicPartition pubSubTopicPartition,
 PubSubPosition startPubSubPosition,
 PubSubPosition endPubSubPosition,
 long numberOfRecords,
 int splitIndex,
 long startIndex)

Method Details

getPubSubTopicPartition

public PubSubTopicPartition getPubSubTopicPartition()

getStartPubSubPosition

public PubSubPosition getStartPubSubPosition()

getEndPubSubPosition

public PubSubPosition getEndPubSubPosition()

getTopicName

public String getTopicName()

getPartitionNumber

public int getPartitionNumber()

getNumberOfRecords

public long getNumberOfRecords()

getSplitIndex

public int getSplitIndex()

getStartIndex

public long getStartIndex()

writeTo

public void writeTo(DataOutput out)
             throws IOException

Throws:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

Throws:

IOException

readFrom

public static PubSubPartitionSplit readFrom(DataInput in)
                                     throws IOException

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

getTopicRepository

public PubSubTopicRepository getTopicRepository()