Package com.linkedin.venice.spark.datawriter.task
package com.linkedin.venice.spark.datawriter.task
-
ClassesClassDescriptionWraps an
Iterator<Row>to count records and accumulate byte sizes as records pass through.All theAccumulatorV2objects that are used in the Spark DataWriter jobs.MaxAccumulator<T extends Comparable<T>>This class is used to track the metrics for the Spark Data Writer task.Holds per-stage diagnostic accumulators for tracking record counts, byte sizes, and timing through the VPJ Spark pipeline.Central registry for per-stage diagnostic metrics in the VPJ Spark pipeline.Wraps anIterator<Row>and records wall-clock timing when the iterator is exhausted.