Package com.linkedin.davinci.kafka.consumer

Class StoreAwarePartitionWiseKafkaConsumerService

java.lang.Object
com.linkedin.venice.service.AbstractVeniceService
com.linkedin.davinci.kafka.consumer.AbstractKafkaConsumerService
com.linkedin.davinci.kafka.consumer.KafkaConsumerService
com.linkedin.davinci.kafka.consumer.PartitionWiseKafkaConsumerService
com.linkedin.davinci.kafka.consumer.StoreAwarePartitionWiseKafkaConsumerService
All Implemented Interfaces:
Closeable, AutoCloseable
public class StoreAwarePartitionWiseKafkaConsumerService extends PartitionWiseKafkaConsumerService
StoreAwarePartitionWiseKafkaConsumerService is used to allocate share consumer from consumer pool at partition granularity. One shared consumer may have multiple topics, and each topic may have multiple consumers. This is store-aware version of topic-wise shared consumer service. The topic partition assignment in this service has a heuristic that we should distribute the all the subscriptions related to a same store / version as even as possible. The load calculation for each consumer will be: Consumer assignment size + IMPOSSIBLE_MAX_PARTITION_COUNT_PER_CONSUMER * subscription count for the same store; and we will pick the least loaded consumer for a new topic partition request. If there is no eligible consumer, it will throw IllegalStateException