Package com.linkedin.venice.spark.input.hdfs

Class VeniceHdfsInputScan

  • All Implemented Interfaces:
    org.apache.spark.sql.connector.read.Batch, org.apache.spark.sql.connector.read.Scan
    public class VeniceHdfsInputScan
extends java.lang.Object
implements org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.connector.read.Batch

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.spark.sql.connector.read.PartitionReaderFactory createReaderFactory()  
      org.apache.spark.sql.connector.read.InputPartition[] planInputPartitions()  
      org.apache.spark.sql.types.StructType readSchema()  
      org.apache.spark.sql.connector.read.Batch toBatch()  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface org.apache.spark.sql.connector.read.Scan

        description, supportedCustomMetrics, toContinuousStream, toMicroBatchStream

    • Constructor Detail

      • VeniceHdfsInputScan

        public VeniceHdfsInputScan​(VeniceProperties jobConfig)

    • Method Detail

      • planInputPartitions

        public org.apache.spark.sql.connector.read.InputPartition[] planInputPartitions()
        Specified by:
        planInputPartitions in interface org.apache.spark.sql.connector.read.Batch

      • createReaderFactory

        public org.apache.spark.sql.connector.read.PartitionReaderFactory createReaderFactory()
        Specified by:
        createReaderFactory in interface org.apache.spark.sql.connector.read.Batch

      • readSchema

        public org.apache.spark.sql.types.StructType readSchema()
        Specified by:
        readSchema in interface org.apache.spark.sql.connector.read.Scan

      • toBatch

        public org.apache.spark.sql.connector.read.Batch toBatch()
        Specified by:
        toBatch in interface org.apache.spark.sql.connector.read.Scan