Package com.linkedin.davinci.client

Class DaVinciRecordTransformer<K,​V,​O>

  • Type Parameters:
    K - type of the input key
    V - type of the input value
    O - type of the output value
    @Experimental
public abstract class DaVinciRecordTransformer<K,​V,​O>
extends java.lang.Object
    This abstract class can be extended in order to transform records stored in the Da Vinci Client. The input is what is consumed from the raw Venice data set, whereas the output is what is stored into Da Vinci's local storage (e.g. RocksDB). Please note that your implementation should be thread safe, and that schema evolution is possible. N.B.: The inputs are wrapped inside Lazy so that if the implementation need not look at them, the deserialization cost is not paid.

    • Constructor Detail

      • DaVinciRecordTransformer

        public DaVinciRecordTransformer​(int storeVersion)

    • Method Detail

      • getKeyOutputSchema

        public abstract org.apache.avro.Schema getKeyOutputSchema()
        This will be the type returned by the DaVinciClient's read operations.
        Returns:
        a Schema corresponding to the type of DaVinciRecordTransformer.

      • getValueOutputSchema

        public abstract org.apache.avro.Schema getValueOutputSchema()

      • put

        public abstract O put​(Lazy<K> key,
                      Lazy<V> value)
        Parameters:
        key - to be put
        value - to be put
        Returns:
        the object to keep in storage, or null if the put should be skipped

      • delete

        public O delete​(Lazy<K> key)
        By default, deletes will proceed. This can be overridden if some deleted records should be kept.
        Parameters:
        key - to be deleted
        Returns:
        the object to keep in storage, or null to proceed with the deletion

      • onStartIngestionTask

        public void onStartIngestionTask()
        This function is called as a lifecycle event at the beginning of an ingestion task. By default, it performs no operation.

      • onEndIngestionTask

        public void onEndIngestionTask()
        This function is called as a lifecycle event at the end of an ingestion task By default, it performs no operation.

      • getValueBytes

        public final java.nio.ByteBuffer getValueBytes​(org.apache.avro.Schema schema,
                                               V value)
        Takes a value, serializes it and wrap it in a ByteByffer.
        Parameters:
        schema - the Avro schema defining the serialization format
        value - value the value to be serialized
        Returns:
        a ByteBuffer containing the serialized value wrapped according to Avro specifications

      • getStoreVersion

        public final int getStoreVersion()
        Returns:
        the storeVersion