Package com.linkedin.venice.hadoop.input.kafka

Class KafkaInputSplit

java.lang.Object
com.linkedin.venice.hadoop.input.kafka.KafkaInputSplit
All Implemented Interfaces:
org.apache.hadoop.io.Writable, org.apache.hadoop.mapred.InputSplit
public class KafkaInputSplit extends Object implements org.apache.hadoop.mapred.InputSplit
We borrowed some idea from the open-sourced attic-crunch lib: https://github.com/apache/attic-crunch/blob/master/crunch-kafka/src/main/java/org/apache/crunch/kafka/record/KafkaInputSplit.java InputSplit that represent retrieving data from a single PubSubTopicPartition between the specified start and end offsets.

  • Constructor Details

    • KafkaInputSplit

      public KafkaInputSplit()
      Nullary constructor for Hadoop to instantiate reflectively.

    • KafkaInputSplit

      public KafkaInputSplit(PubSubPartitionSplit pubSubPartitionSplit)
      Original constructor signature, now wiring through to PubSubPartitionSplit.

  • Method Details

    • getTopicPartition

      public PubSubTopicPartition getTopicPartition()

    • getStartingOffset

      public PubSubPosition getStartingOffset()

    • getEndingOffset

      public PubSubPosition getEndingOffset()

    • getNumberOfRecords

      public long getNumberOfRecords()

    • getLength

      public long getLength()
      Specified by:
      getLength in interface org.apache.hadoop.mapred.InputSplit

    • getLocations

      public String[] getLocations()
      Specified by:
      getLocations in interface org.apache.hadoop.mapred.InputSplit

    • write

      public void write(DataOutput out) throws IOException
      Specified by:
      write in interface org.apache.hadoop.io.Writable
      Throws:
      IOException

    • readFields

      public void readFields(DataInput in) throws IOException
      Specified by:
      readFields in interface org.apache.hadoop.io.Writable
      Throws:
      IOException

    • toString

      public String toString()
      Overrides:
      toString in class Object