Package com.linkedin.venice.pubsub.adapter.kafka

Class TopicPartitionsOffsetsTracker

  • public class TopicPartitionsOffsetsTracker
extends java.lang.Object
    This class tracks consumed topic partitions' offsets

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void clearAllOffsetState()
      Clear all tracked offsets state
      long getEndOffset​(java.lang.String topic, int partition)
      Get the end offset of a topic partition
      long getOffsetLag​(java.lang.String topic, int partition)
      Get consuming offset lag on a topic partition
      void removeTrackedOffsets​(org.apache.kafka.common.TopicPartition topicPartition)
      Remove tracked offsets state of a topic partition.
      void updateEndAndCurrentOffsets​(org.apache.kafka.clients.consumer.ConsumerRecords<byte[],​byte[]> records, org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> kafkaConsumer)
      Update the end and current offsets of consumed partitions based on consumer metrics and given consumed records.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • TopicPartitionsOffsetsTracker

        public TopicPartitionsOffsetsTracker()

    • Method Detail

      • updateEndAndCurrentOffsets

        public void updateEndAndCurrentOffsets​(org.apache.kafka.clients.consumer.ConsumerRecords<byte[],​byte[]> records,
                                       org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> kafkaConsumer)
        Update the end and current offsets of consumed partitions based on consumer metrics and given consumed records. For each topic partition, end offset == consumed current offset + offset lag and current offset is determined to be the largest offset amongst the records passed in.
        Parameters:
        records - consumed records
        kafkaConsumer - from which to extract metrics

      • removeTrackedOffsets

        public void removeTrackedOffsets​(org.apache.kafka.common.TopicPartition topicPartition)
        Remove tracked offsets state of a topic partition.
        Parameters:
        topicPartition -

      • clearAllOffsetState

        public void clearAllOffsetState()
        Clear all tracked offsets state

      • getEndOffset

        public long getEndOffset​(java.lang.String topic,
                         int partition)
        Get the end offset of a topic partition
        Parameters:
        topic -
        partition -
        Returns:
        end offset of a topic partition if there is any.

      • getOffsetLag

        public long getOffsetLag​(java.lang.String topic,
                         int partition)
        Get consuming offset lag on a topic partition
        Parameters:
        topic -
        partition -
        Returns:
        end offset of a topic partition if there is any.