Package com.linkedin.venice.hadoop.input.kafka.ttl

Class VeniceKafkaInputTTLFilter

java.lang.Object

com.linkedin.venice.hadoop.AbstractVeniceFilter<INPUT_VALUE>

com.linkedin.venice.hadoop.input.kafka.ttl.VeniceRmdTTLFilter<KafkaInputMapperValue>

com.linkedin.venice.hadoop.input.kafka.ttl.VeniceKafkaInputTTLFilter

All Implemented Interfaces:

Closeable, AutoCloseable

public class VeniceKafkaInputTTLFilter
extends VeniceRmdTTLFilter<KafkaInputMapperValue>

This class is responsible to filter records based on the RMD information and the ttl config by taking KIF input KafkaInputMapperValue.

Field Summary

Fields inherited from class com.linkedin.venice.hadoop.input.kafka.ttl.VeniceRmdTTLFilter

rmdSchemaMap, valueSchemaMap

Constructor Summary

Constructors

Constructor	Description
VeniceKafkaInputTTLFilter(VeniceProperties props)

Method Summary

Modifier and Type	Method	Description
protected ByteBuffer	getRmdPayload(KafkaInputMapperValue kafkaInputMapperValue)
protected int	getRmdProtocolId(KafkaInputMapperValue kafkaInputMapperValue)
protected int	getSchemaId(KafkaInputMapperValue kafkaInputMapperValue)
protected ByteBuffer	getValuePayload(KafkaInputMapperValue kafkaInputMapperValue)
protected boolean	skipRmdRecord(KafkaInputMapperValue input)	When schemeId is negative, it indicates a chunked record.
protected void	updateRmdPayload(KafkaInputMapperValue kafkaInputMapperValue, ByteBuffer payload)
protected void	updateValuePayload(KafkaInputMapperValue kafkaInputMapperValue, byte[] payload)

Methods inherited from class com.linkedin.venice.hadoop.input.kafka.ttl.VeniceRmdTTLFilter

checkAndMaybeFilterValue, close

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VeniceKafkaInputTTLFilter

public VeniceKafkaInputTTLFilter(VeniceProperties props)
                          throws IOException

Throws:

IOException

Method Details

getSchemaId

protected int getSchemaId(KafkaInputMapperValue kafkaInputMapperValue)

Specified by:

getSchemaId in class VeniceRmdTTLFilter<KafkaInputMapperValue>

getRmdProtocolId

protected int getRmdProtocolId(KafkaInputMapperValue kafkaInputMapperValue)

Specified by:

getRmdProtocolId in class VeniceRmdTTLFilter<KafkaInputMapperValue>

getRmdPayload

protected ByteBuffer getRmdPayload(KafkaInputMapperValue kafkaInputMapperValue)

Specified by:

getRmdPayload in class VeniceRmdTTLFilter<KafkaInputMapperValue>

getValuePayload

protected ByteBuffer getValuePayload(KafkaInputMapperValue kafkaInputMapperValue)

Specified by:

getValuePayload in class VeniceRmdTTLFilter<KafkaInputMapperValue>

updateRmdPayload

protected void updateRmdPayload(KafkaInputMapperValue kafkaInputMapperValue,
 ByteBuffer payload)

Specified by:

updateRmdPayload in class VeniceRmdTTLFilter<KafkaInputMapperValue>

updateValuePayload

protected void updateValuePayload(KafkaInputMapperValue kafkaInputMapperValue,
 byte[] payload)

Specified by:

updateValuePayload in class VeniceRmdTTLFilter<KafkaInputMapperValue>

skipRmdRecord

protected boolean skipRmdRecord(KafkaInputMapperValue input)

When schemeId is negative, it indicates a chunked record. Skip it and pass it to Reducer as chunk will only be re-assembled at Reducer.

Overrides:

skipRmdRecord in class VeniceRmdTTLFilter<KafkaInputMapperValue>

Parameters:

input -

Returns: