Package com.linkedin.venice.hadoop.mapreduce.datawriter.task

Class ReporterBackedMapReduceDataWriterTaskTracker

java.lang.Object

com.linkedin.venice.hadoop.mapreduce.datawriter.task.ReporterBackedMapReduceDataWriterTaskTracker

All Implemented Interfaces:

DataWriterTaskTracker, TaskTracker

public class ReporterBackedMapReduceDataWriterTaskTracker
extends Object
implements DataWriterTaskTracker

An interface to report and retrieve metrics related to data writer tasks during the execution of a MapReduce job.

Field Summary

Fields inherited from interface com.linkedin.venice.hadoop.task.TaskTracker

PROGRESS_COMPLETED, PROGRESS_NOT_SUPPORTED

Constructor Summary

Constructors

Constructor	Description
ReporterBackedMapReduceDataWriterTaskTracker(org.apache.hadoop.mapred.Reporter reporter)

Method Summary

Modifier and Type	Method	Description
long	getDuplicateKeyWithDistinctValueCount()
float	getProgress()	Get the progress of the task.
long	getRecordTooLargeFailureCount()
org.apache.hadoop.mapred.Reporter	getReporter()
long	getTotalKeySize()
long	getTotalPutOrDeleteRecordsCount()
long	getTotalValueSize()
long	getUncompressedRecordTooLargeFailureCount()
long	getWriteAclAuthorizationFailureCount()
void	heartbeat()	Report task heartbeat if the execution engine requires it
void	trackCompressedValueSize(int size)
void	trackDuplicateKeyWithDistinctValue(int count)
void	trackDuplicateKeyWithIdenticalValue(int count)
void	trackEmptyRecord()
void	trackGzipCompressedValueSize(int size)
void	trackKeySize(int size)
void	trackPartitionWriterClose()
void	trackPutOrDeleteRecord()
void	trackRecordSentToPubSub()
void	trackRecordTooLargeFailure()
void	trackRepushTtlFilteredRecord()
void	trackSprayAllPartitions()
void	trackUncompressedRecordTooLargeFailure()
void	trackUncompressedValueSize(int size)
void	trackWriteAclAuthorizationFailure()
void	trackZstdCompressedValueSize(int size)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.venice.hadoop.task.datawriter.DataWriterTaskTracker

getLargestUncompressedValueSize, getOutputRecordsCount, getPartitionWriterCloseCount, getRepushTtlFilterCount, getSprayAllPartitionsCount, getTotalGzipCompressedValueSize, getTotalUncompressedValueSize, getTotalZstdCompressedValueSize, trackLargestUncompressedValueSize

Constructor Details

ReporterBackedMapReduceDataWriterTaskTracker

public ReporterBackedMapReduceDataWriterTaskTracker(org.apache.hadoop.mapred.Reporter reporter)

Method Details

getReporter

public org.apache.hadoop.mapred.Reporter getReporter()

heartbeat

public void heartbeat()

Description copied from interface: TaskTracker

Report task heartbeat if the execution engine requires it

Specified by:

heartbeat in interface TaskTracker

getProgress

public float getProgress()

Description copied from interface: TaskTracker

Get the progress of the task. If the engine supports fetching the progress, progress is represented as a number between 0 and 1 (inclusive). If the engine doesn't support fetching the progress, a value of -1 is returned.

Specified by:

getProgress in interface TaskTracker

trackSprayAllPartitions

public void trackSprayAllPartitions()

Specified by:

trackSprayAllPartitions in interface DataWriterTaskTracker

trackEmptyRecord

public void trackEmptyRecord()

Specified by:

trackEmptyRecord in interface DataWriterTaskTracker

trackKeySize

public void trackKeySize(int size)

Specified by:

trackKeySize in interface DataWriterTaskTracker

trackUncompressedValueSize

public void trackUncompressedValueSize(int size)

Specified by:

trackUncompressedValueSize in interface DataWriterTaskTracker

trackCompressedValueSize

public void trackCompressedValueSize(int size)

Specified by:

trackCompressedValueSize in interface DataWriterTaskTracker

trackGzipCompressedValueSize

public void trackGzipCompressedValueSize(int size)

Specified by:

trackGzipCompressedValueSize in interface DataWriterTaskTracker

trackZstdCompressedValueSize

public void trackZstdCompressedValueSize(int size)

Specified by:

trackZstdCompressedValueSize in interface DataWriterTaskTracker

trackWriteAclAuthorizationFailure

public void trackWriteAclAuthorizationFailure()

Specified by:

trackWriteAclAuthorizationFailure in interface DataWriterTaskTracker

trackRecordTooLargeFailure

public void trackRecordTooLargeFailure()

Specified by:

trackRecordTooLargeFailure in interface DataWriterTaskTracker

trackUncompressedRecordTooLargeFailure

public void trackUncompressedRecordTooLargeFailure()

Specified by:

trackUncompressedRecordTooLargeFailure in interface DataWriterTaskTracker

trackRecordSentToPubSub

public void trackRecordSentToPubSub()

Specified by:

trackRecordSentToPubSub in interface DataWriterTaskTracker

trackDuplicateKeyWithDistinctValue

public void trackDuplicateKeyWithDistinctValue(int count)

Specified by:

trackDuplicateKeyWithDistinctValue in interface DataWriterTaskTracker

trackDuplicateKeyWithIdenticalValue

public void trackDuplicateKeyWithIdenticalValue(int count)

Specified by:

trackDuplicateKeyWithIdenticalValue in interface DataWriterTaskTracker

trackRepushTtlFilteredRecord

public void trackRepushTtlFilteredRecord()

Specified by:

trackRepushTtlFilteredRecord in interface DataWriterTaskTracker

trackPartitionWriterClose

public void trackPartitionWriterClose()

Specified by:

trackPartitionWriterClose in interface DataWriterTaskTracker

trackPutOrDeleteRecord

public void trackPutOrDeleteRecord()

Specified by:

trackPutOrDeleteRecord in interface DataWriterTaskTracker

getTotalKeySize

public long getTotalKeySize()

Specified by:

getTotalKeySize in interface DataWriterTaskTracker

getTotalValueSize

public long getTotalValueSize()

Specified by:

getTotalValueSize in interface DataWriterTaskTracker

getRecordTooLargeFailureCount

public long getRecordTooLargeFailureCount()

Specified by:

getRecordTooLargeFailureCount in interface DataWriterTaskTracker

getUncompressedRecordTooLargeFailureCount

public long getUncompressedRecordTooLargeFailureCount()

Specified by:

getUncompressedRecordTooLargeFailureCount in interface DataWriterTaskTracker

getWriteAclAuthorizationFailureCount

public long getWriteAclAuthorizationFailureCount()

Specified by:

getWriteAclAuthorizationFailureCount in interface DataWriterTaskTracker

getDuplicateKeyWithDistinctValueCount

public long getDuplicateKeyWithDistinctValueCount()

Specified by:

getDuplicateKeyWithDistinctValueCount in interface DataWriterTaskTracker

getTotalPutOrDeleteRecordsCount

public long getTotalPutOrDeleteRecordsCount()

Specified by:

getTotalPutOrDeleteRecordsCount in interface DataWriterTaskTracker