Package com.linkedin.davinci.storage.chunking

Interface ChunkingAdapter<CHUNKS_CONTAINER,VALUE>

All Known Implementing Classes:
AbstractAvroChunkingAdapter, BatchGetChunkingAdapter, GenericChunkingAdapter, GenericRecordChunkingAdapter, RawBytesChunkingAdapter, SingleGetChunkingAdapter, SpecificRecordChunkingAdapter
public interface ChunkingAdapter<CHUNKS_CONTAINER,VALUE>
This interface allows each code path which needs to interact with chunked values to optimize the way the re-assembly is handled so that the final form in which the is consumed is generated as efficiently as possible, via the use of a temporary ChunkingAdapter.

  • Method Details

    • constructValue

      default VALUE constructValue(byte[] fullBytes, int bytesLength, VALUE reusedValue, org.apache.avro.io.BinaryDecoder reusedDecoder, ReadResponseStats responseStats, int writerSchemaId, int readerSchemaId, StoreDeserializerCache<VALUE> storeDeserializerCache, VeniceCompressor compressor)
      Used to wrap a small value fetched from the storage engine into the right type of class needed by the query code.

      The following parameters are mandatory:

      Parameters:
      fullBytes - includes both the schema ID header and the payload

      The following parameters can be ignored, by implementing constructValue(int, byte[]):

      bytesLength -
      reusedValue - a previous instance of to be re-used in order to minimize GC
      reusedDecoder - a previous instance of BinaryDecoder to be re-used in order to minimize GC
      responseStats - the ReadResponseStats which carries certain metrics to be recorded at the end
      writerSchemaId - schema used to serialize the value
      readerSchemaId -

    • constructValue

      default VALUE constructValue(byte[] valueOnlyBytes, int offset, int bytesLength, RecordDeserializer<VALUE> recordDeserializer, VeniceCompressor veniceCompressor)

    • constructValue

      default VALUE constructValue(int schemaId, byte[] fullBytes)
      This function can be implemented by the adapters which need fewer parameters.

    • addChunkIntoContainer

      void addChunkIntoContainer(CHUNKS_CONTAINER container, int chunkIndex, byte[] valueChunk)
      Used to incrementally add a into the container.

    • constructChunksContainer

      CHUNKS_CONTAINER constructChunksContainer(ChunkedValueManifest chunkedValueManifest)
      Used to construct the right kind of container (according to the query code) to hold a large value which needs to be incrementally re-assembled from many smaller chunks.

    • constructValue

      default VALUE constructValue(CHUNKS_CONTAINER chunksContainer, VALUE reusedValue, org.apache.avro.io.BinaryDecoder reusedDecoder, ReadResponseStats responseStats, int writerSchemaId, int readerSchemaId, StoreDeserializerCache<VALUE> storeDeserializerCache, VeniceCompressor compressor)
      Used to wrap a large value re-assembled with the use of a into the right type of class needed by the query code.

      The following parameters are mandatory:

      Parameters:
      chunksContainer - temporary , obtained from constructChunksContainer(ChunkedValueManifest)

      The following parameters can be ignored, by implementing constructValue(int, Object):

      reusedValue - a previous instance of to be re-used in order to minimize GC
      reusedDecoder - a previous instance of BinaryDecoder to be re-used in order to minimize GC
      responseStats - the ReadResponseStats which carries certain metrics to be recorded at the end
      writerSchemaId - of the user's value
      readerSchemaId -

    • constructValue

      default VALUE constructValue(int schemaId, CHUNKS_CONTAINER chunksContainer)
      This function can be implemented by the adapters which need fewer parameters.