Package com.linkedin.davinci.kafka.consumer

Class InactiveTopicPartitionChecker

java.lang.Object

com.linkedin.venice.service.AbstractVeniceService

com.linkedin.davinci.kafka.consumer.InactiveTopicPartitionChecker

All Implemented Interfaces:

Closeable, AutoCloseable

public class InactiveTopicPartitionChecker
extends AbstractVeniceService

A service that monitors Kafka topic partitions for inactivity and automatically pauses/resumes them to prevent blocking of active topic partitions.

This checker runs periodically to:

• Identify topic partitions that haven't been successfully polled within a configured threshold
• Pause inactive topic partitions to prevent them from blocking consumption of active topic partitions
• Resume previously paused topic partitions that have become active again

The service maintains state about which topic partitions are currently paused for each consumer and makes intelligent decisions about when to pause/resume based on the current activity status.

This is particularly useful in scenarios where some topic partitions may become temporarily unavailable or slow, preventing the consumer from making progress on other healthy topic partitions.

The checker uses per-topic-partition statistics from ConsumptionTask to determine activity, specifically looking at the last successful poll timestamp for each topic partition.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService

AbstractVeniceService.ServiceState

Field Summary

Fields inherited from class com.linkedin.venice.service.AbstractVeniceService

logger, serviceState

Constructor Summary

Constructors

Constructor	Description
InactiveTopicPartitionChecker(IndexedMap<com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer,com.linkedin.davinci.kafka.consumer.ConsumptionTask> consumerToConsumptionTask, long inactiveTopicPartitionCheckIntervalInSeconds, long inactiveTopicPartitionThresholdInSeconds)	Creates a new InactiveTopicPartitionChecker.

Method Summary

Modifier and Type	Method	Description
boolean	startInner()	Starts the inactive topic partition checker service.
void	stopInner()	Stops the inactive topic partition checker service.

Methods inherited from class com.linkedin.venice.service.AbstractVeniceService

close, getName, isRunning, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

InactiveTopicPartitionChecker

public InactiveTopicPartitionChecker(IndexedMap<com.linkedin.davinci.kafka.consumer.SharedKafkaConsumer,com.linkedin.davinci.kafka.consumer.ConsumptionTask> consumerToConsumptionTask,
 long inactiveTopicPartitionCheckIntervalInSeconds,
 long inactiveTopicPartitionThresholdInSeconds)

Creates a new InactiveTopicPartitionChecker.

Parameters:

consumerToConsumptionTask - map of consumers to their consumption tasks, used to check topic partition activity status

inactiveTopicPartitionCheckIntervalInSeconds - how often to check for inactive topic partitions (in milliseconds)

inactiveTopicPartitionThresholdInSeconds - threshold after which a topic partition is considered inactive (in milliseconds)

Method Details

startInner

public boolean startInner()
                   throws Exception

Starts the inactive topic partition checker service.

This method schedules a periodic task that will run every inactiveTopicPartitionCheckIntervalInMs milliseconds to check for and handle inactive topic partitions.

Specified by:

startInner in class AbstractVeniceService

Returns:

true if the service started successfully

Throws:

Exception - if there's an error starting the service

stopInner

public void stopInner()
               throws Exception

Stops the inactive topic partition checker service.

This method gracefully shuts down the scheduled executor service, waiting up to 5 seconds for running tasks to complete before forcing shutdown.

Specified by:

stopInner in class AbstractVeniceService

Throws:

Exception - if there's an error stopping the service