Class VeniceHdfsInputTable

  • All Implemented Interfaces:
    org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.Table

    public class VeniceHdfsInputTable
    extends java.lang.Object
    implements org.apache.spark.sql.connector.catalog.SupportsRead
    A table format that is used by Spark to read Avro files from HDFS for use in VenicePushJob.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.util.Set<org.apache.spark.sql.connector.catalog.TableCapability> capabilities()  
      java.lang.String name()  
      org.apache.spark.sql.connector.read.ScanBuilder newScanBuilder​(org.apache.spark.sql.util.CaseInsensitiveStringMap options)  
      org.apache.spark.sql.types.StructType schema()  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
      • Methods inherited from interface org.apache.spark.sql.connector.catalog.Table

        partitioning, properties
    • Constructor Detail

      • VeniceHdfsInputTable

        public VeniceHdfsInputTable​(VeniceProperties jobConfig)
    • Method Detail

      • newScanBuilder

        public org.apache.spark.sql.connector.read.ScanBuilder newScanBuilder​(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
        Specified by:
        newScanBuilder in interface org.apache.spark.sql.connector.catalog.SupportsRead
      • name

        public java.lang.String name()
        Specified by:
        name in interface org.apache.spark.sql.connector.catalog.Table
      • schema

        public org.apache.spark.sql.types.StructType schema()
        Specified by:
        schema in interface org.apache.spark.sql.connector.catalog.Table
      • capabilities

        public java.util.Set<org.apache.spark.sql.connector.catalog.TableCapability> capabilities()
        Specified by:
        capabilities in interface org.apache.spark.sql.connector.catalog.Table