Class VeniceRmdTTLFilter<INPUT_VALUE>

  • Type Parameters:
    INPUT_VALUE - , the value contains schemaID, rmdId and rmdPayload that are required to retrieve RMD timestamp.
    All Implemented Interfaces:, java.lang.AutoCloseable
    Direct Known Subclasses:
    VeniceChunkedPayloadTTLFilter, VeniceKafkaInputTTLFilter

    public abstract class VeniceRmdTTLFilter<INPUT_VALUE>
    extends AbstractVeniceFilter<INPUT_VALUE>
    This class is responsible to filter records based on the RMD information and the ttl config. It requires RMD schemas for a given store from an existing HDFS directory to be able to parse timestamp information in RMD.
    • Field Detail

      • rmdSchemaMap

        protected final java.util.Map<RmdVersionId,​org.apache.avro.Schema> rmdSchemaMap
      • valueSchemaMap

        protected final java.util.Map<java.lang.Integer,​org.apache.avro.Schema> valueSchemaMap
    • Constructor Detail

      • VeniceRmdTTLFilter

        public VeniceRmdTTLFilter​(VeniceProperties props)
    • Method Detail

      • checkAndMaybeFilterValue

        public boolean checkAndMaybeFilterValue​(INPUT_VALUE value)
        Description copied from class: AbstractVeniceFilter
        This function implements how to parse the value and determine if filtering is needed. For certain value from Active/Active partial update enabled stores, it might filter out part of its input value and only keep the remaining fresh part based on filter timestamp.
        Specified by:
        checkAndMaybeFilterValue in class AbstractVeniceFilter<INPUT_VALUE>
        true if the value should be filtered out, otherwise false.
      • close

        public void close()
      • getSchemaId

        protected abstract int getSchemaId​(INPUT_VALUE value)
      • getRmdProtocolId

        protected abstract int getRmdProtocolId​(INPUT_VALUE value)
      • getRmdPayload

        protected abstract java.nio.ByteBuffer getRmdPayload​(INPUT_VALUE value)
      • getValuePayload

        protected abstract java.nio.ByteBuffer getValuePayload​(INPUT_VALUE value)
      • updateRmdPayload

        protected abstract void updateRmdPayload​(INPUT_VALUE value,
                                                 java.nio.ByteBuffer payload)
      • updateValuePayload

        protected abstract void updateValuePayload​(INPUT_VALUE value,
                                                   byte[] payload)
      • skipRmdRecord

        protected boolean skipRmdRecord​(INPUT_VALUE value)
        Define how records could be skipped if certain conditions are met. Do not skip by default.
        value -
        true if this record should not be filtered and skipped.