Package com.linkedin.davinci.consumer

Class LocalBootstrappingVeniceChangelogConsumer<K,V>

java.lang.Object

com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl<K,V>

com.linkedin.davinci.consumer.VeniceAfterImageConsumerImpl<K,V>

com.linkedin.davinci.consumer.LocalBootstrappingVeniceChangelogConsumer<K,V>

Type Parameters:

K -

V -

All Implemented Interfaces:

BootstrappingVeniceChangelogConsumer<K,V>, VeniceChangelogConsumer<K,V>

public class LocalBootstrappingVeniceChangelogConsumer<K,V>
extends VeniceAfterImageConsumerImpl<K,V>

This is a wrapper class on top of InternalBootstrappingVeniceChangelogConsumerImpl. This confines the usage of this class for clients to the methods exposed on the interface. This is meant to prevent users from doing seek() calls which would render the local state inconsistent.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl

VeniceChangelogConsumerImpl.HeartbeatReporterThread

Field Summary

Fields inherited from class com.linkedin.davinci.consumer.VeniceAfterImageConsumerImpl

versionSwapDetectionIntervalTimeInMs

Fields inherited from class com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl

changeCaptureStats, changelogClientConfig, chunkAssembler, compressorFactory, compressorMap, currentValuePayloadSize, currentVersionHighWatermarks, currentVersionLastHeartbeat, heartbeatReporterThread, isSubscribed, keyDeserializer, NO_OP_COMPRESSOR, partitionCount, partitionToBootstrapState, partitionToDeleteMessageCount, partitionToPutMessageCount, pubSubConsumer, pubSubTopicRepository, replicationMetadataSchemaRepository, rmdDeserializerCache, schemaReader, specificValueClass, startTimestamp, storeDeserializerCache, storeName, storeRepository, subscribeTime, userEventChunkingAdapter

Constructor Summary

Constructors

Constructor	Description
LocalBootstrappingVeniceChangelogConsumer(ChangelogClientConfig changelogClientConfig, PubSubConsumerAdapter pubSubConsumer, String consumerId)

Method Summary

Modifier and Type	Method	Description
protected boolean	handleVersionSwapControlMessage(ControlMessage controlMessage, PubSubTopicPartition pubSubTopicPartition, String topicSuffix)
protected Collection<PubSubMessage<K,ChangeEvent<V>,VeniceChangeCoordinate>>	internalPoll(long timeoutInMs, String topicSuffix)
protected <T> T	processRecordBytes(ByteBuffer decompressedBytes, T deserializedValue, byte[] key, ByteBuffer value, PubSubTopicPartition partition, int readerSchemaId, long recordOffset)
CompletableFuture<Void>	seekWithBootStrap(Set<Integer> partitions)
CompletableFuture<Void>	start()
CompletableFuture<Void>	start(Set<Integer> partitions)	Start performs both a topic subscription and catch up.
void	stop()

Methods inherited from class com.linkedin.davinci.consumer.VeniceAfterImageConsumerImpl

internalSeek, poll, seekToEndOfPush, seekToTail, seekToTimestamps, subscribe, subscribed

Methods inherited from class com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl

close, convertPubSubMessageToPubSubChangeEventMessage, extractOffsetVectorFromMessage, getChangeCaptureStats, getChangelogClientConfig, getHeartbeatReporterThread, getLatestCoordinate, getPartitionCount, getPubSubConsumer, getSubscribeTime, getTopicAssignment, getTopicPartition, getVersionCompressor, handleControlMessage, internalPoll, internalSeekToTail, internalSeekToTimestamps, internalSubscribe, isCaughtUp, pause, pause, resume, resume, seekToBeginningOfPush, seekToBeginningOfPush, seekToCheckpoint, seekToEndOfPush, seekToTail, seekToTimestamp, setStoreRepository, subscribeAll, switchToNewTopic, unsubscribe, unsubscribeAll

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.davinci.consumer.BootstrappingVeniceChangelogConsumer

poll

Constructor Details

LocalBootstrappingVeniceChangelogConsumer

public LocalBootstrappingVeniceChangelogConsumer(ChangelogClientConfig changelogClientConfig,
 PubSubConsumerAdapter pubSubConsumer,
 String consumerId)

Method Details

handleVersionSwapControlMessage

protected boolean handleVersionSwapControlMessage(ControlMessage controlMessage,
 PubSubTopicPartition pubSubTopicPartition,
 String topicSuffix)

Overrides:

handleVersionSwapControlMessage in class VeniceChangelogConsumerImpl<K,V>

internalPoll

protected Collection<PubSubMessage<K,ChangeEvent<V>,VeniceChangeCoordinate>> internalPoll(long timeoutInMs,
 String topicSuffix)

Overrides:

internalPoll in class VeniceChangelogConsumerImpl<K,V>

processRecordBytes

protected <T> T processRecordBytes(ByteBuffer decompressedBytes,
 T deserializedValue,
 byte[] key,
 ByteBuffer value,
 PubSubTopicPartition partition,
 int readerSchemaId,
 long recordOffset)

Overrides:

processRecordBytes in class VeniceChangelogConsumerImpl<K,V>

seekWithBootStrap

public CompletableFuture<Void> seekWithBootStrap(Set<Integer> partitions)

start

public CompletableFuture<Void> start(Set<Integer> partitions)

Description copied from interface: BootstrappingVeniceChangelogConsumer

Start performs both a topic subscription and catch up. The client will look at the latest offset in the server and sync bootstrap data up to that point in changes. Once that is done for all partitions, the future will complete. NOTE: This future may take some time to complete depending on how much data needs to be ingested in order to catch up with the time that this client started.

Specified by:

start in interface BootstrappingVeniceChangelogConsumer<K,V>

Parameters:

partitions - which partition id's to catch up with

Returns:

a future that completes once catch up is complete for all passed in partitions.

start

public CompletableFuture<Void> start()

Specified by:

start in interface BootstrappingVeniceChangelogConsumer<K,V>

stop

public void stop()
          throws Exception

Specified by:

stop in interface BootstrappingVeniceChangelogConsumer<K,V>

Throws:

Exception