Package com.linkedin.davinci.kafka.consumer

Class AbstractKafkaConsumerService

java.lang.Object

com.linkedin.venice.service.AbstractVeniceService

com.linkedin.davinci.kafka.consumer.AbstractKafkaConsumerService

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

KafkaConsumerService, KafkaConsumerServiceDelegator

public abstract class AbstractKafkaConsumerService
extends AbstractVeniceService

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService

AbstractVeniceService.ServiceState

Field Summary

Fields inherited from class com.linkedin.venice.service.AbstractVeniceService

logger, serviceState

Constructor Summary

Constructors

Constructor	Description
AbstractKafkaConsumerService()

Method Summary

Modifier and Type	Method	Description
abstract com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer	assignConsumerFor(PubSubTopic versionTopic, PubSubTopicPartition topicPartition)
abstract void	batchUnsubscribe(PubSubTopic versionTopic, Set<PubSubTopicPartition> topicPartitionsToUnSub)
abstract com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer	getConsumerAssignedToVersionTopicPartition(PubSubTopic versionTopic, PubSubTopicPartition topicPartition)
abstract Map<PubSubTopicPartition,TopicPartitionIngestionInfo>	getIngestionInfoFor(PubSubTopic versionTopic, PubSubTopicPartition pubSubTopicPartition, boolean respectRedundantLoggingFilter)	This is for providing ingestion related information for a specific topic partition from the implementation of this class.
abstract long	getLatestOffsetBasedOnMetrics(PubSubTopic versionTopic, PubSubTopicPartition pubSubTopicPartition)
abstract long	getMaxElapsedTimeMSSinceLastPollInConsumerPool()
abstract Map<PubSubTopicPartition,Long>	getStaleTopicPartitions(long thresholdTimestamp)
abstract boolean	hasAnySubscriptionFor(PubSubTopic versionTopic)
abstract void	startConsumptionIntoDataReceiver(PartitionReplicaIngestionContext partitionReplicaIngestionContext, PubSubPosition lastReadPosition, ConsumedDataReceiver<List<DefaultPubSubMessage>> consumedDataReceiver)
void	unSubscribe(PubSubTopic versionTopic, PubSubTopicPartition pubSubTopicPartition)
abstract void	unSubscribe(PubSubTopic versionTopic, PubSubTopicPartition pubSubTopicPartition, long timeoutMs)
abstract void	unsubscribeAll(PubSubTopic versionTopic)

Methods inherited from class com.linkedin.venice.service.AbstractVeniceService

close, getName, isRunning, start, startInner, stop, stopInner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

AbstractKafkaConsumerService

public AbstractKafkaConsumerService()

Method Details

getConsumerAssignedToVersionTopicPartition

public abstract com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer getConsumerAssignedToVersionTopicPartition(PubSubTopic versionTopic,
 PubSubTopicPartition topicPartition)

assignConsumerFor

public abstract com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer assignConsumerFor(PubSubTopic versionTopic,
 PubSubTopicPartition topicPartition)

unsubscribeAll

public abstract void unsubscribeAll(PubSubTopic versionTopic)

unSubscribe

public void unSubscribe(PubSubTopic versionTopic,
 PubSubTopicPartition pubSubTopicPartition)

unSubscribe

public abstract void unSubscribe(PubSubTopic versionTopic,
 PubSubTopicPartition pubSubTopicPartition,
 long timeoutMs)

batchUnsubscribe

public abstract void batchUnsubscribe(PubSubTopic versionTopic,
 Set<PubSubTopicPartition> topicPartitionsToUnSub)

hasAnySubscriptionFor

public abstract boolean hasAnySubscriptionFor(PubSubTopic versionTopic)

getMaxElapsedTimeMSSinceLastPollInConsumerPool

public abstract long getMaxElapsedTimeMSSinceLastPollInConsumerPool()

startConsumptionIntoDataReceiver

public abstract void startConsumptionIntoDataReceiver(PartitionReplicaIngestionContext partitionReplicaIngestionContext,
 PubSubPosition lastReadPosition,
 ConsumedDataReceiver<List<DefaultPubSubMessage>> consumedDataReceiver)

getLatestOffsetBasedOnMetrics

public abstract long getLatestOffsetBasedOnMetrics(PubSubTopic versionTopic,
 PubSubTopicPartition pubSubTopicPartition)

getIngestionInfoFor

public abstract Map<PubSubTopicPartition,TopicPartitionIngestionInfo> getIngestionInfoFor(PubSubTopic versionTopic,
 PubSubTopicPartition pubSubTopicPartition,
 boolean respectRedundantLoggingFilter)

This is for providing ingestion related information for a specific topic partition from the implementation of this class.

Parameters:

respectRedundantLoggingFilter - here is to guide if we need to prepare the info map, set to true when calling from heartbeat monitoring to enable rate-limiting; set to false for admin commands or tests where all info is needed.

getStaleTopicPartitions

public abstract Map<PubSubTopicPartition,Long> getStaleTopicPartitions(long thresholdTimestamp)