Package com.linkedin.venice.spark.input

Class VeniceAbstractPartitionReader

java.lang.Object

com.linkedin.venice.spark.input.VeniceAbstractPartitionReader

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>

Direct Known Subclasses:

VeniceHdfsInputPartitionReader

public abstract class VeniceAbstractPartitionReader
extends Object
implements org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>

Constructor Summary

Constructors

Constructor	Description
VeniceAbstractPartitionReader(VeniceProperties jobConfig, org.apache.spark.sql.connector.read.InputPartition partition)

Method Summary

Modifier and Type	Method	Description
void	close()
protected abstract VeniceRecordIterator	createRecordIterator(VeniceProperties jobConfig, org.apache.spark.sql.connector.read.InputPartition partition)
org.apache.spark.sql.catalyst.InternalRow	get()
boolean	next()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.connector.read.PartitionReader

currentMetricsValues

Constructor Details

VeniceAbstractPartitionReader

public VeniceAbstractPartitionReader(VeniceProperties jobConfig,
 org.apache.spark.sql.connector.read.InputPartition partition)

Method Details

createRecordIterator

protected abstract VeniceRecordIterator createRecordIterator(VeniceProperties jobConfig,
 org.apache.spark.sql.connector.read.InputPartition partition)

next

public boolean next()
             throws IOException

Specified by:

next in interface org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>

Throws:

IOException

get

public org.apache.spark.sql.catalyst.InternalRow get()

Specified by:

get in interface org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException