Package com.linkedin.venice.pubsub.adapter.kafka

Class TopicPartitionsOffsetsTracker

java.lang.Object
com.linkedin.venice.pubsub.adapter.kafka.TopicPartitionsOffsetsTracker
public class TopicPartitionsOffsetsTracker extends Object
This class tracks consumed topic partitions' offsets

  • Constructor Details

    • TopicPartitionsOffsetsTracker

      public TopicPartitionsOffsetsTracker()

  • Method Details

    • updateEndAndCurrentOffsets

      public void updateEndAndCurrentOffsets(org.apache.kafka.clients.consumer.ConsumerRecords<byte[],byte[]> records, org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> kafkaConsumer)
      Update the end and current offsets of consumed partitions based on consumer metrics and given consumed records. For each topic partition, end offset == consumed current offset + offset lag and current offset is determined to be the largest offset amongst the records passed in.
      Parameters:
      records - consumed records
      kafkaConsumer - from which to extract metrics

    • removeTrackedOffsets

      public void removeTrackedOffsets(org.apache.kafka.common.TopicPartition topicPartition)
      Remove tracked offsets state of a topic partition.
      Parameters:
      topicPartition -

    • clearAllOffsetState

      public void clearAllOffsetState()
      Clear all tracked offsets state

    • getEndOffset

      public long getEndOffset(String topic, int partition)
      Get the end offset of a topic partition
      Parameters:
      topic -
      partition -
      Returns:
      end offset of a topic partition if there is any.

    • getOffsetLag

      public long getOffsetLag(String topic, int partition)
      Get consuming offset lag on a topic partition
      Parameters:
      topic -
      partition -
      Returns:
      end offset of a topic partition if there is any.