Class PartitionWiseKafkaConsumerService
- java.lang.Object
-
- com.linkedin.venice.service.AbstractVeniceService
-
- com.linkedin.davinci.kafka.consumer.AbstractKafkaConsumerService
-
- com.linkedin.davinci.kafka.consumer.KafkaConsumerService
-
- com.linkedin.davinci.kafka.consumer.PartitionWiseKafkaConsumerService
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
public class PartitionWiseKafkaConsumerService extends KafkaConsumerService
PartitionWiseKafkaConsumerService
is used to allocate share consumer from consumer pool at partition granularity. One shared consumer may have multiple topics, and each topic may have multiple consumers. For this basic implementation, we rely on round-robin to allocate next consumer from pool to achieve efficient and balanced shared consumer partition assignment load. We can improve this allocation strategy if we need to.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class com.linkedin.davinci.kafka.consumer.KafkaConsumerService
KafkaConsumerService.ConsumerAssignmentStrategy
-
Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService
AbstractVeniceService.ServiceState
-
-
Field Summary
-
Fields inherited from class com.linkedin.davinci.kafka.consumer.KafkaConsumerService
aggStats, consumerToConsumptionTask, kafkaUrl, kafkaUrlForLogger, versionTopicToTopicPartitionToConsumer
-
Fields inherited from class com.linkedin.venice.service.AbstractVeniceService
logger, serviceState
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer
pickConsumerForPartition(PubSubTopic versionTopic, PubSubTopicPartition topicPartition)
-
Methods inherited from class com.linkedin.davinci.kafka.consumer.KafkaConsumerService
assignConsumerFor, batchUnsubscribe, getConsumerAssignedToVersionTopicPartition, getIngestionInfoFromConsumer, getLatestOffsetBasedOnMetrics, getMaxElapsedTimeMSSinceLastPollInConsumerPool, getOffsetLagBasedOnMetrics, hasAnySubscriptionFor, removeTopicPartitionFromConsumptionTask, startConsumptionIntoDataReceiver, startInner, stopInner, unSubscribe, unsubscribeAll
-
-
-
-
Method Detail
-
pickConsumerForPartition
protected com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer pickConsumerForPartition(PubSubTopic versionTopic, PubSubTopicPartition topicPartition)
- Specified by:
pickConsumerForPartition
in classKafkaConsumerService
-
-