Package com.linkedin.venice.hadoop.spark.datawriter.recordprocessor

Class SparkInputRecordProcessorFactory

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.spark.api.java.function.FlatMapFunction<org.apache.spark.sql.Row,​org.apache.spark.sql.Row>
    public class SparkInputRecordProcessorFactory
extends java.lang.Object
implements org.apache.spark.api.java.function.FlatMapFunction<org.apache.spark.sql.Row,​org.apache.spark.sql.Row>
    A Factory class to create individual SparkInputRecordProcessor for processing records in a Spark Dataframe.
    See Also:
    Serialized Form

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.util.Iterator<org.apache.spark.sql.Row> call​(org.apache.spark.sql.Row row)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • SparkInputRecordProcessorFactory

        public SparkInputRecordProcessorFactory​(org.apache.spark.broadcast.Broadcast<java.util.Properties> jobProps,
                                        DataWriterAccumulators accumulators)

    • Method Detail

      • call

        public java.util.Iterator<org.apache.spark.sql.Row> call​(org.apache.spark.sql.Row row)
                                                  throws java.lang.Exception
        Specified by:
        call in interface org.apache.spark.api.java.function.FlatMapFunction<org.apache.spark.sql.Row,​org.apache.spark.sql.Row>
        Throws:
        java.lang.Exception