Package com.linkedin.venice.spark.input.hdfs

Class VeniceHdfsInputTable

  • All Implemented Interfaces:
    org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.Table
    public class VeniceHdfsInputTable
extends java.lang.Object
implements org.apache.spark.sql.connector.catalog.SupportsRead
    A table format that is used by Spark to read Avro files from HDFS for use in VenicePushJob.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.util.Set<org.apache.spark.sql.connector.catalog.TableCapability> capabilities()  
      java.lang.String name()  
      org.apache.spark.sql.connector.read.ScanBuilder newScanBuilder​(org.apache.spark.sql.util.CaseInsensitiveStringMap options)  
      org.apache.spark.sql.types.StructType schema()  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface org.apache.spark.sql.connector.catalog.Table

        partitioning, properties

    • Constructor Detail

      • VeniceHdfsInputTable

        public VeniceHdfsInputTable​(VeniceProperties jobConfig)

    • Method Detail

      • newScanBuilder

        public org.apache.spark.sql.connector.read.ScanBuilder newScanBuilder​(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
        Specified by:
        newScanBuilder in interface org.apache.spark.sql.connector.catalog.SupportsRead

      • name

        public java.lang.String name()
        Specified by:
        name in interface org.apache.spark.sql.connector.catalog.Table

      • schema

        public org.apache.spark.sql.types.StructType schema()
        Specified by:
        schema in interface org.apache.spark.sql.connector.catalog.Table

      • capabilities

        public java.util.Set<org.apache.spark.sql.connector.catalog.TableCapability> capabilities()
        Specified by:
        capabilities in interface org.apache.spark.sql.connector.catalog.Table