Package com.linkedin.venice.spark.input.pubsub

Class SparkPubSubInputFormat

java.lang.Object

com.linkedin.venice.spark.input.pubsub.SparkPubSubInputFormat

All Implemented Interfaces:

org.apache.spark.sql.connector.read.Batch, org.apache.spark.sql.connector.read.Scan

public class SparkPubSubInputFormat
extends Object
implements org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.connector.read.Batch

Constructor Summary

Constructors

Constructor	Description
SparkPubSubInputFormat(VeniceProperties jobConfig)

Method Summary

Modifier and Type	Method	Description
org.apache.spark.sql.connector.read.PartitionReaderFactory	createReaderFactory()
org.apache.spark.sql.connector.read.InputPartition[]	planInputPartitions()
org.apache.spark.sql.types.StructType	readSchema()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.connector.read.Scan

description, supportedCustomMetrics, toBatch, toContinuousStream, toMicroBatchStream

Constructor Details

SparkPubSubInputFormat

public SparkPubSubInputFormat(VeniceProperties jobConfig)

Method Details

planInputPartitions

public org.apache.spark.sql.connector.read.InputPartition[] planInputPartitions()

Specified by:

planInputPartitions in interface org.apache.spark.sql.connector.read.Batch

createReaderFactory

public org.apache.spark.sql.connector.read.PartitionReaderFactory createReaderFactory()

Specified by:

createReaderFactory in interface org.apache.spark.sql.connector.read.Batch

readSchema

public org.apache.spark.sql.types.StructType readSchema()

Specified by:

readSchema in interface org.apache.spark.sql.connector.read.Scan