Package com.linkedin.venice.spark.datawriter.writer

Class SparkPartitionWriterFactory

java.lang.Object

com.linkedin.venice.spark.datawriter.writer.SparkPartitionWriterFactory

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.MapPartitionsFunction<org.apache.spark.sql.Row,org.apache.spark.sql.Row>

public class SparkPartitionWriterFactory
extends Object
implements org.apache.spark.api.java.function.MapPartitionsFunction<org.apache.spark.sql.Row,org.apache.spark.sql.Row>

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
SparkPartitionWriterFactory(org.apache.spark.broadcast.Broadcast<Properties> jobProps, DataWriterAccumulators accumulators)

Method Summary

Modifier and Type	Method	Description
Iterator<org.apache.spark.sql.Row>	call(Iterator<org.apache.spark.sql.Row> rows)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SparkPartitionWriterFactory

public SparkPartitionWriterFactory(org.apache.spark.broadcast.Broadcast<Properties> jobProps,
 DataWriterAccumulators accumulators)

Method Details

call

public Iterator<org.apache.spark.sql.Row> call(Iterator<org.apache.spark.sql.Row> rows)
                                        throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.MapPartitionsFunction<org.apache.spark.sql.Row,org.apache.spark.sql.Row>

Throws:

Exception