Package com.linkedin.venice.hadoop
Class KafkaInputDataInfoProvider
- java.lang.Object
-
- com.linkedin.venice.hadoop.KafkaInputDataInfoProvider
-
- All Implemented Interfaces:
InputDataInfoProvider
,java.io.Closeable
,java.lang.AutoCloseable
public class KafkaInputDataInfoProvider extends java.lang.Object implements InputDataInfoProvider
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface com.linkedin.venice.hadoop.InputDataInfoProvider
InputDataInfoProvider.InputDataInfo
-
-
Constructor Summary
Constructors Constructor Description KafkaInputDataInfoProvider()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
org.apache.avro.Schema
extractAvroSubSchema(org.apache.avro.Schema origin, java.lang.String fieldName)
long
getInputLastModificationTime(java.lang.String inputUri)
void
initZstdConfig(int numFiles)
byte[]
trainZstdDictionary()
InputDataInfoProvider.InputDataInfo
validateInputAndGetInfo(java.lang.String inputUri)
-
-
-
Method Detail
-
validateInputAndGetInfo
public InputDataInfoProvider.InputDataInfo validateInputAndGetInfo(java.lang.String inputUri) throws java.lang.Exception
- Specified by:
validateInputAndGetInfo
in interfaceInputDataInfoProvider
- Throws:
java.lang.Exception
-
initZstdConfig
public void initZstdConfig(int numFiles)
- Specified by:
initZstdConfig
in interfaceInputDataInfoProvider
-
trainZstdDictionary
public byte[] trainZstdDictionary()
- Specified by:
trainZstdDictionary
in interfaceInputDataInfoProvider
-
extractAvroSubSchema
public org.apache.avro.Schema extractAvroSubSchema(org.apache.avro.Schema origin, java.lang.String fieldName)
- Specified by:
extractAvroSubSchema
in interfaceInputDataInfoProvider
-
getInputLastModificationTime
public long getInputLastModificationTime(java.lang.String inputUri) throws java.io.IOException
- Specified by:
getInputLastModificationTime
in interfaceInputDataInfoProvider
- Throws:
java.io.IOException
-
close
public void close() throws java.io.IOException
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Throws:
java.io.IOException
-
-