Package com.linkedin.venice.spark.datawriter.task

Class DataWriterAccumulators

  • All Implemented Interfaces:
    java.io.Serializable
    public class DataWriterAccumulators
extends java.lang.Object
implements java.io.Serializable
    All the AccumulatorV2 objects that are used in the Spark DataWriter jobs.
    See Also:
    Serialized Form

    • Field Detail

      • sprayAllPartitionsTriggeredCount

        public final org.apache.spark.util.LongAccumulator sprayAllPartitionsTriggeredCount

      • emptyRecordCounter

        public final org.apache.spark.util.LongAccumulator emptyRecordCounter

      • totalKeySizeCounter

        public final org.apache.spark.util.LongAccumulator totalKeySizeCounter

      • uncompressedValueSizeCounter

        public final org.apache.spark.util.LongAccumulator uncompressedValueSizeCounter

      • compressedValueSizeCounter

        public final org.apache.spark.util.LongAccumulator compressedValueSizeCounter

      • gzipCompressedValueSizeCounter

        public final org.apache.spark.util.LongAccumulator gzipCompressedValueSizeCounter

      • zstdCompressedValueSizeCounter

        public final org.apache.spark.util.LongAccumulator zstdCompressedValueSizeCounter

      • outputRecordCounter

        public final org.apache.spark.util.LongAccumulator outputRecordCounter

      • duplicateKeyWithIdenticalValueCounter

        public final org.apache.spark.util.LongAccumulator duplicateKeyWithIdenticalValueCounter

      • writeAclAuthorizationFailureCounter

        public final org.apache.spark.util.LongAccumulator writeAclAuthorizationFailureCounter

      • recordTooLargeFailureCounter

        public final org.apache.spark.util.LongAccumulator recordTooLargeFailureCounter

      • duplicateKeyWithDistinctValueCounter

        public final org.apache.spark.util.LongAccumulator duplicateKeyWithDistinctValueCounter

      • partitionWriterCloseCounter

        public final org.apache.spark.util.LongAccumulator partitionWriterCloseCounter

      • repushTtlFilteredRecordCounter

        public final org.apache.spark.util.LongAccumulator repushTtlFilteredRecordCounter

    • Constructor Detail

      • DataWriterAccumulators

        public DataWriterAccumulators​(org.apache.spark.sql.SparkSession session)