Package com.linkedin.venice.hadoop.task.datawriter

Interface DataWriterTaskTracker

All Superinterfaces:
TaskTracker
All Known Implementing Classes:
CounterBackedMapReduceDataWriterTaskTracker, ReporterBackedMapReduceDataWriterTaskTracker, SparkDataWriterTaskTracker
public interface DataWriterTaskTracker extends TaskTracker
An interface to report and retrieve metrics related to data writer tasks.

  • Method Details

    • trackSprayAllPartitions

      default void trackSprayAllPartitions()

    • trackEmptyRecord

      default void trackEmptyRecord()

    • trackKeySize

      default void trackKeySize(int size)

    • trackUncompressedValueSize

      default void trackUncompressedValueSize(int size)

    • trackLargestUncompressedValueSize

      default void trackLargestUncompressedValueSize(int size)
      This accumulator performs a 'max' operation, which is not natively supported by either Spark or Hadoop. It is implemented using a custom accumulator in Spark.

    • trackCompressedValueSize

      default void trackCompressedValueSize(int size)

    • trackGzipCompressedValueSize

      default void trackGzipCompressedValueSize(int size)

    • trackZstdCompressedValueSize

      default void trackZstdCompressedValueSize(int size)

    • trackWriteAclAuthorizationFailure

      default void trackWriteAclAuthorizationFailure()

    • trackRecordTooLargeFailure

      default void trackRecordTooLargeFailure()

    • trackUncompressedRecordTooLargeFailure

      default void trackUncompressedRecordTooLargeFailure()

    • trackRecordSentToPubSub

      default void trackRecordSentToPubSub()

    • trackDuplicateKeyWithDistinctValue

      default void trackDuplicateKeyWithDistinctValue(int count)

    • trackDuplicateKeyWithIdenticalValue

      default void trackDuplicateKeyWithIdenticalValue(int count)

    • trackRepushTtlFilteredRecord

      default void trackRepushTtlFilteredRecord()

    • trackPutOrDeleteRecord

      default void trackPutOrDeleteRecord()

    • trackPartitionWriterClose

      default void trackPartitionWriterClose()

    • getSprayAllPartitionsCount

      default long getSprayAllPartitionsCount()

    • getTotalKeySize

      default long getTotalKeySize()

    • getTotalValueSize

      default long getTotalValueSize()

    • getTotalUncompressedValueSize

      default long getTotalUncompressedValueSize()

    • getLargestUncompressedValueSize

      default int getLargestUncompressedValueSize()
      This accumulator performs a 'max' operation, which is not natively supported by either Spark or Hadoop. It is implemented using a custom accumulator in Spark.

    • getTotalGzipCompressedValueSize

      default long getTotalGzipCompressedValueSize()

    • getTotalZstdCompressedValueSize

      default long getTotalZstdCompressedValueSize()

    • getRecordTooLargeFailureCount

      default long getRecordTooLargeFailureCount()

    • getUncompressedRecordTooLargeFailureCount

      default long getUncompressedRecordTooLargeFailureCount()

    • getWriteAclAuthorizationFailureCount

      default long getWriteAclAuthorizationFailureCount()

    • getDuplicateKeyWithDistinctValueCount

      default long getDuplicateKeyWithDistinctValueCount()

    • getOutputRecordsCount

      default long getOutputRecordsCount()

    • getPartitionWriterCloseCount

      default long getPartitionWriterCloseCount()

    • getRepushTtlFilterCount

      default long getRepushTtlFilterCount()

    • getTotalPutOrDeleteRecordsCount

      default long getTotalPutOrDeleteRecordsCount()