Package com.linkedin.venice.spark.input.hdfs

Class VeniceHdfsInputScan

java.lang.Object
com.linkedin.venice.spark.input.hdfs.VeniceHdfsInputScan
All Implemented Interfaces:
org.apache.spark.sql.connector.read.Batch, org.apache.spark.sql.connector.read.Scan
public class VeniceHdfsInputScan extends Object implements org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.connector.read.Batch

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.sql.connector.read.Scan

    org.apache.spark.sql.connector.read.Scan.ColumnarSupportMode

  • Constructor Summary

    Constructors
    Constructor
    Description
    VeniceHdfsInputScan(VeniceProperties jobConfig)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.sql.connector.read.PartitionReaderFactory
    createReaderFactory()
     
    org.apache.spark.sql.connector.read.InputPartition[]
    planInputPartitions()
     
    org.apache.spark.sql.types.StructType
    readSchema()
     
    org.apache.spark.sql.connector.read.Batch
    toBatch()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.connector.read.Scan

    columnarSupportMode, description, reportDriverMetrics, supportedCustomMetrics, toContinuousStream, toMicroBatchStream

  • Constructor Details

  • Method Details

    • planInputPartitions

      public org.apache.spark.sql.connector.read.InputPartition[] planInputPartitions()
      Specified by:
      planInputPartitions in interface org.apache.spark.sql.connector.read.Batch

    • createReaderFactory

      public org.apache.spark.sql.connector.read.PartitionReaderFactory createReaderFactory()
      Specified by:
      createReaderFactory in interface org.apache.spark.sql.connector.read.Batch

    • readSchema

      public org.apache.spark.sql.types.StructType readSchema()
      Specified by:
      readSchema in interface org.apache.spark.sql.connector.read.Scan

    • toBatch

      public org.apache.spark.sql.connector.read.Batch toBatch()
      Specified by:
      toBatch in interface org.apache.spark.sql.connector.read.Scan