Package com.linkedin.venice.spark.datawriter.task


package com.linkedin.venice.spark.datawriter.task
  • Classes
    Class
    Description
    Wraps an Iterator<Row> to count records and accumulate byte sizes as records pass through.
    All the AccumulatorV2 objects that are used in the Spark DataWriter jobs.
     
    This class is used to track the metrics for the Spark Data Writer task.
    Holds per-stage diagnostic accumulators for tracking record counts, byte sizes, and timing through the VPJ Spark pipeline.
    Central registry for per-stage diagnostic metrics in the VPJ Spark pipeline.
    Wraps an Iterator<Row> and records wall-clock timing when the iterator is exhausted.