Package com.linkedin.venice.spark.consistency

Class LilyPadSnapshotBuilder

java.lang.Object

com.linkedin.venice.spark.consistency.LilyPadSnapshotBuilder

public final class LilyPadSnapshotBuilder
extends Object

Builds a LilyPadUtils.Snapshot from a PubSub partition split by reading records through a PubSubSplitIterator and wrapping positions in ComparablePubSubPosition.

This class is the bridge between the PubSub infrastructure and the generic lily-pad algorithm in LilyPadUtils. It is the only class that depends on PubSub types for snapshot construction.

Method Summary

Modifier and Type	Method	Description
static LilyPadUtils.Snapshot<ComparablePubSubPosition>	buildSnapshot(PubSubSplitIterator iterator, TopicManager topicManager, PubSubPositionDeserializer pubSubPositionDeserializer, int numberOfRegions)	Iterates over a pre-built PubSubSplitIterator and returns a per-key history of PUT and DELETE records in the order they appeared in the VT partition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

buildSnapshot

public static LilyPadUtils.Snapshot<ComparablePubSubPosition> buildSnapshot(PubSubSplitIterator iterator,
 TopicManager topicManager,
 PubSubPositionDeserializer pubSubPositionDeserializer,
 int numberOfRegions)

Iterates over a pre-built PubSubSplitIterator and returns a per-key history of PUT and DELETE records in the order they appeared in the VT partition.

The caller is responsible for constructing the iterator (including consumer creation and position discovery) and for closing it after this method returns.

The running high-watermark is computed in VT position order, so each LilyPadUtils.KeyRecord's highWatermark reflects global RT progress up to that point.

Parameters:

iterator - a ready-to-read iterator over a single VT partition split

topicManager - TopicManager used for position comparison (max watermark)

pubSubPositionDeserializer - deserializer for upstream PubSubPosition bytes

numberOfRegions - total number of regions in the AA topology (determines position vector size)

Returns:

LilyPadUtils.Snapshot containing the per-key record map and the final partition high-watermark