Package com.linkedin.venice.hadoop

Class KafkaInputDataInfoProvider

java.lang.Object

com.linkedin.venice.hadoop.KafkaInputDataInfoProvider

All Implemented Interfaces:

InputDataInfoProvider, Closeable, AutoCloseable

public class KafkaInputDataInfoProvider
extends Object
implements InputDataInfoProvider

Nested Class Summary

Nested classes/interfaces inherited from interface com.linkedin.venice.hadoop.InputDataInfoProvider

InputDataInfoProvider.InputDataInfo

Constructor Summary

Constructors

Constructor	Description
KafkaInputDataInfoProvider()

Method Summary

Modifier and Type	Method	Description
void	close()
org.apache.avro.Schema	extractAvroSubSchema(org.apache.avro.Schema origin, String fieldName)
long	getInputLastModificationTime(String inputUri)
PushJobZstdConfig	initZstdConfig(int numFiles)
byte[]	trainZstdDictionary()
InputDataInfoProvider.InputDataInfo	validateInputAndGetInfo(String inputUri)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

KafkaInputDataInfoProvider

public KafkaInputDataInfoProvider()

Method Details

validateInputAndGetInfo

public InputDataInfoProvider.InputDataInfo validateInputAndGetInfo(String inputUri)
                                                            throws Exception

Specified by:

validateInputAndGetInfo in interface InputDataInfoProvider

Throws:

Exception

initZstdConfig

public PushJobZstdConfig initZstdConfig(int numFiles)

Specified by:

initZstdConfig in interface InputDataInfoProvider

trainZstdDictionary

public byte[] trainZstdDictionary()

Specified by:

trainZstdDictionary in interface InputDataInfoProvider

extractAvroSubSchema

public org.apache.avro.Schema extractAvroSubSchema(org.apache.avro.Schema origin,
 String fieldName)

Specified by:

extractAvroSubSchema in interface InputDataInfoProvider

getInputLastModificationTime

public long getInputLastModificationTime(String inputUri)
                                  throws IOException

Specified by:

getInputLastModificationTime in interface InputDataInfoProvider

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException