Package com.linkedin.venice.kafka.validation

Class Segment

java.lang.Object

com.linkedin.venice.kafka.validation.Segment

public class Segment
extends Object

A segment is a sequence of messages sent by a single producer into a single partition. The same producer will maintain a different segment in each of the partitions it writes into. On the other hand, many distinct producers can maintain their own segment for the same partition, in which case, the messages contained in these various segments will be interleaved. This class keeps track of the state of a segment: - The partition it belongs to. - Its segmentNumber number within its partition. - Whether it has started. - Whether it has ended. - The current sequence number. - The running checksum.

Constructor Summary

Constructors

Constructor	Description
Segment(int partition, int segmentNumber, int sequenceNumber, CheckSumType checkSumType, Map<CharSequence,CharSequence> debugInfo, Map<CharSequence,Long> aggregates)
Segment(int partition, int segmentNumber, CheckSumType checkSumType)
Segment(int partition, ProducerPartitionState state)	Build a segment with checkpoint producer state on disk.
Segment(Segment segment)

Method Summary

Modifier and Type	Method	Description
boolean	addToCheckSum(KafkaKey key, KafkaMessageEnvelope messageEnvelope)	This function updates the running checksum as follows, depending on the MessageType: 1.
void	end(boolean finalSegment)
boolean	equals(Object obj)
Map<CharSequence,Long>	getAggregates()
int	getAndIncrementSequenceNumber()	N.B.
byte[]	getCheckSumState()	`synchronized` keyword will guarantee the caller will always get the checksum after processing the full record in function: addToCheckSum(KafkaKey, KafkaMessageEnvelope).
CheckSumType	getCheckSumType()
Map<CharSequence,CharSequence>	getDebugInfo()
byte[]	getFinalCheckSum()
long	getLastRecordProducerTimestamp()
long	getLastRecordTimestamp()
long	getLastSuccessfulOffset()
int	getPartition()
int	getSegmentNumber()
int	getSequenceNumber()
SegmentStatus	getStatus()
int	hashCode()
boolean	isEnded()
boolean	isNewSegment()
boolean	isRegistered()
boolean	isStarted()
void	registeredSegment()
void	setLastRecordProducerTimestamp(long lastRecordProducerTimestamp)
void	setLastRecordTimestamp(long lastRecordTimestamp)
void	setLastSuccessfulOffset(long lastSuccessfulOffset)
void	setNewSegment(boolean newSegment)
void	setSequenceNumber(int sequenceNum)	This function should only be used for hybrid store after receiving 'EOP' when skipping a message in the sequence.
void	setStarted(boolean started)
void	start()
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

Segment

public Segment(int partition,
 int segmentNumber,
 int sequenceNumber,
 CheckSumType checkSumType,
 Map<CharSequence,CharSequence> debugInfo,
 Map<CharSequence,Long> aggregates)

Segment

public Segment(int partition,
 int segmentNumber,
 CheckSumType checkSumType)

Segment

public Segment(int partition,
 ProducerPartitionState state)

Build a segment with checkpoint producer state on disk.

Segment

public Segment(Segment segment)

Method Details

getSegmentNumber

public int getSegmentNumber()

getPartition

public int getPartition()

getAndIncrementSequenceNumber

public int getAndIncrementSequenceNumber()

N.B. This function is not threadsafe. Locking must be handled by the caller.

setSequenceNumber

public void setSequenceNumber(int sequenceNum)

This function should only be used for hybrid store after receiving 'EOP' when skipping a message in the sequence.

Parameters:

sequenceNum -

getSequenceNumber

public int getSequenceNumber()

getDebugInfo

public Map<CharSequence,CharSequence> getDebugInfo()

getAggregates

public Map<CharSequence,Long> getAggregates()

getCheckSumState

public byte[] getCheckSumState()

`synchronized` keyword will guarantee the caller will always get the checksum after processing the full record in function: addToCheckSum(KafkaKey, KafkaMessageEnvelope).

Returns:

getCheckSumType

public CheckSumType getCheckSumType()

isStarted

public boolean isStarted()

isEnded

public boolean isEnded()

isRegistered

public boolean isRegistered()

getLastSuccessfulOffset

public long getLastSuccessfulOffset()

setLastSuccessfulOffset

public void setLastSuccessfulOffset(long lastSuccessfulOffset)

getLastRecordTimestamp

public long getLastRecordTimestamp()

setLastRecordTimestamp

public void setLastRecordTimestamp(long lastRecordTimestamp)

getLastRecordProducerTimestamp

public long getLastRecordProducerTimestamp()

setLastRecordProducerTimestamp

public void setLastRecordProducerTimestamp(long lastRecordProducerTimestamp)

start

public void start()

end

public void end(boolean finalSegment)

registeredSegment

public void registeredSegment()

isNewSegment

public boolean isNewSegment()

setNewSegment

public void setNewSegment(boolean newSegment)

addToCheckSum

public boolean addToCheckSum(KafkaKey key,
 KafkaMessageEnvelope messageEnvelope)
                      throws UnsupportedMessageTypeException

This function updates the running checksum as follows, depending on the MessageType: 1. MessageType.CONTROL_MESSAGE, depending on the specific type: 1.1. ControlMessageType.END_OF_SEGMENT: No-op. 1.2. All others: Message type, control message type. 2. MessageType.PUT: Message type, key, schema ID and value. 3. MessageType.DELETE: Message type, key. Both Producer and Consumer should use this same function in order to ensure coherent behavior.

Parameters:

key - of the message to add into the running checksum

messageEnvelope - to add into the running checksum

Returns:

true if something was added to checksum, false otherwise (which happens when hitting an ControlMessageType.END_OF_SEGMENT).

Throws:

UnsupportedMessageTypeException - if the MessageType or ControlMessageType is unknown.

getFinalCheckSum

public byte[] getFinalCheckSum()

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

toString

public String toString()

Overrides:

toString in class Object

getStatus

public SegmentStatus getStatus()

setStarted

public void setStarted(boolean started)