Package com.linkedin.davinci.consumer

Class LocalBootstrappingVeniceChangelogConsumer<K,V>

java.lang.Object

com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl<K,V>

com.linkedin.davinci.consumer.VeniceAfterImageConsumerImpl<K,V>

com.linkedin.davinci.consumer.LocalBootstrappingVeniceChangelogConsumer<K,V>

Type Parameters:

K -

V -

All Implemented Interfaces:

BootstrappingVeniceChangelogConsumer<K,V>, VeniceChangelogConsumer<K,V>

public class LocalBootstrappingVeniceChangelogConsumer<K,V>
extends VeniceAfterImageConsumerImpl<K,V>

This is a wrapper class on top of InternalBootstrappingVeniceChangelogConsumerImpl. This confines the usage of this class for clients to the methods exposed on the interface. This is meant to prevent users from doing seek() calls which would render the local state inconsistent.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl

VeniceChangelogConsumerImpl.HeartbeatReporterThread

Field Summary

Fields inherited from class com.linkedin.davinci.consumer.VeniceAfterImageConsumerImpl

versionSwapDetectionIntervalTimeInMs

Fields inherited from class com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl

changeCaptureStats, changelogClientConfig, chunkAssembler, compressorFactory, compressorMap, currentValuePayloadSize, currentVersionHighWatermarks, currentVersionLastHeartbeat, heartbeatReporterThread, keyDeserializer, NO_OP_COMPRESSOR, partitionCount, partitionToBootstrapState, partitionToDeleteMessageCount, partitionToPutMessageCount, pubSubConsumer, pubSubTopicRepository, replicationMetadataSchemaRepository, rmdDeserializerCache, schemaReader, specificValueClass, startTimestamp, storeDeserializerCache, storeName, storeRepository, subscribeTime, userEventChunkingAdapter

Constructor Summary

Constructors

Constructor	Description
LocalBootstrappingVeniceChangelogConsumer(ChangelogClientConfig changelogClientConfig, PubSubConsumerAdapter pubSubConsumer, java.lang.String consumerId)

Method Summary

Modifier and Type	Method	Description
protected boolean	handleVersionSwapControlMessage(ControlMessage controlMessage, PubSubTopicPartition pubSubTopicPartition, java.lang.String topicSuffix)
protected java.util.Collection<PubSubMessage<K,ChangeEvent<V>,VeniceChangeCoordinate>>	internalPoll(long timeoutInMs, java.lang.String topicSuffix)
protected <T> T	processRecordBytes(java.nio.ByteBuffer decompressedBytes, T deserializedValue, byte[] key, java.nio.ByteBuffer value, PubSubTopicPartition partition, int readerSchemaId, long recordOffset)
java.util.concurrent.CompletableFuture<java.lang.Void>	seekWithBootStrap(java.util.Set<java.lang.Integer> partitions)
java.util.concurrent.CompletableFuture<java.lang.Void>	start()
java.util.concurrent.CompletableFuture<java.lang.Void>	start(java.util.Set<java.lang.Integer> partitions)	Start performs both a topic subscription and catch up.
void	stop()

Methods inherited from class com.linkedin.davinci.consumer.VeniceAfterImageConsumerImpl

internalSeek, poll, seekToEndOfPush, seekToTail, seekToTimestamps, subscribe

Methods inherited from class com.linkedin.davinci.consumer.VeniceChangelogConsumerImpl

close, convertPubSubMessageToPubSubChangeEventMessage, extractOffsetVectorFromMessage, getChangeCaptureStats, getChangelogClientConfig, getHeartbeatReporterThread, getLatestCoordinate, getPartitionCount, getPubSubConsumer, getSubscribeTime, getTopicAssignment, getTopicPartition, getVersionCompressor, handleControlMessage, internalPoll, internalSeekToTail, internalSeekToTimestamps, internalSubscribe, isCaughtUp, pause, pause, resume, resume, seekToBeginningOfPush, seekToBeginningOfPush, seekToCheckpoint, seekToEndOfPush, seekToTail, seekToTimestamp, setStoreRepository, subscribeAll, switchToNewTopic, unsubscribe, unsubscribeAll

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.davinci.consumer.BootstrappingVeniceChangelogConsumer

poll

Constructor Detail

LocalBootstrappingVeniceChangelogConsumer

public LocalBootstrappingVeniceChangelogConsumer(ChangelogClientConfig changelogClientConfig,
                                                 PubSubConsumerAdapter pubSubConsumer,
                                                 java.lang.String consumerId)

Method Detail

handleVersionSwapControlMessage

protected boolean handleVersionSwapControlMessage(ControlMessage controlMessage,
                                                  PubSubTopicPartition pubSubTopicPartition,
                                                  java.lang.String topicSuffix)

Overrides:

handleVersionSwapControlMessage in class VeniceChangelogConsumerImpl<K,V>

internalPoll

protected java.util.Collection<PubSubMessage<K,ChangeEvent<V>,VeniceChangeCoordinate>> internalPoll(long timeoutInMs,
                                                                                                                java.lang.String topicSuffix)

Overrides:

internalPoll in class VeniceChangelogConsumerImpl<K,V>

processRecordBytes

protected <T> T processRecordBytes(java.nio.ByteBuffer decompressedBytes,
                                   T deserializedValue,
                                   byte[] key,
                                   java.nio.ByteBuffer value,
                                   PubSubTopicPartition partition,
                                   int readerSchemaId,
                                   long recordOffset)

Overrides:

processRecordBytes in class VeniceChangelogConsumerImpl<K,V>

seekWithBootStrap

public java.util.concurrent.CompletableFuture<java.lang.Void> seekWithBootStrap(java.util.Set<java.lang.Integer> partitions)

start

public java.util.concurrent.CompletableFuture<java.lang.Void> start(java.util.Set<java.lang.Integer> partitions)

Description copied from interface: BootstrappingVeniceChangelogConsumer

Start performs both a topic subscription and catch up. The client will look at the latest offset in the server and sync bootstrap data up to that point in changes. Once that is done for all partitions, the future will complete. NOTE: This future may take some time to complete depending on how much data needs to be ingested in order to catch up with the time that this client started.

Specified by:

start in interface BootstrappingVeniceChangelogConsumer<K,V>

Parameters:

partitions - which partition id's to catch up with

Returns:

a future that completes once catch up is complete for all passed in partitions.

start

public java.util.concurrent.CompletableFuture<java.lang.Void> start()

Specified by:

start in interface BootstrappingVeniceChangelogConsumer<K,V>

stop

public void stop()
          throws java.lang.Exception

Specified by:

stop in interface BootstrappingVeniceChangelogConsumer<K,V>

Throws:

java.lang.Exception