Package com.linkedin.venice

Class KafkaTopicDumper

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    public class KafkaTopicDumper
extends java.lang.Object
implements java.lang.AutoCloseable
    This class contains logic to dump Venice Kafka topics. It has several modes: (1) Log metadata: Print out Kafka message metadata to console. (2) Log data record: Print out data record's key/value and optionally RMD to console. For now it only supports version topic and realtime topic. (3) Save data record to file: If both (1)&(2) is not enabled, it will save all the data record value payload to local disk.

    • Constructor Summary

      Constructors 
      Constructor Description
      KafkaTopicDumper​(ControllerClient controllerClient, PubSubConsumerAdapter consumer, java.lang.String topic, int partitionNumber, long startingOffset, int messageCount, java.lang.String parentDir, int maxConsumeAttempts, boolean logMetadata, boolean logDataRecord, boolean logRmdRecord)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      int fetchAndProcess()
      1.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • KafkaTopicDumper

        public KafkaTopicDumper​(ControllerClient controllerClient,
                        PubSubConsumerAdapter consumer,
                        java.lang.String topic,
                        int partitionNumber,
                        long startingOffset,
                        int messageCount,
                        java.lang.String parentDir,
                        int maxConsumeAttempts,
                        boolean logMetadata,
                        boolean logDataRecord,
                        boolean logRmdRecord)

    • Method Detail

      • fetchAndProcess

        public int fetchAndProcess()
        1. Fetch up to messageCount messages in this partition. 2. Discard non-control messages.

      • close

        public void close()
           throws java.lang.Exception
        Specified by:
        close in interface java.lang.AutoCloseable
        Throws:
        java.lang.Exception