Package com.linkedin.davinci.utils

Class ChunkAssembler

java.lang.Object

com.linkedin.davinci.utils.ChunkAssembler

Direct Known Subclasses:

InMemoryChunkAssembler, RocksDBChunkAssembler

public abstract class ChunkAssembler
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
protected final StorageEngine	bufferStorageEngine

Constructor Summary

Constructors

Constructor	Description
ChunkAssembler(StorageEngine bufferStorageEngine, boolean skipFailedToAssembleRecords)

Method Summary

Modifier and Type	Method	Description
ByteBufferValueRecord<ByteBuffer>	bufferAndAssembleRecord(PubSubTopicPartition pubSubTopicPartition, int schemaId, byte[] keyBytes, ByteBuffer valueBytes, PubSubPosition recordOffset, VeniceCompressor compressor)	Buffers and assembles chunks of a record.
void	clearBuffer()
static ByteBuffer	decompressValueIfNeeded(ByteBuffer value, int schemaId, VeniceCompressor compressor)	For chunked records, the chunk assembler already decompresses the assembled value via RawBytesChunkingAdapter's decompressingInputStreamDecoder.
static boolean	isChunkedRecord(int schemaId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

bufferStorageEngine

protected final StorageEngine bufferStorageEngine

Constructor Details

ChunkAssembler

public ChunkAssembler(StorageEngine bufferStorageEngine,
 boolean skipFailedToAssembleRecords)

Method Details

bufferAndAssembleRecord

public ByteBufferValueRecord<ByteBuffer> bufferAndAssembleRecord(PubSubTopicPartition pubSubTopicPartition,
 int schemaId,
 byte[] keyBytes,
 ByteBuffer valueBytes,
 PubSubPosition recordOffset,
 VeniceCompressor compressor)

Buffers and assembles chunks of a record. If the record is chunked, it stores the chunks and returns null. Once all chunks of a record are received, it returns the compressed and serialized assembled record.

clearBuffer

public void clearBuffer()

isChunkedRecord

public static boolean isChunkedRecord(int schemaId)

decompressValueIfNeeded

public static ByteBuffer decompressValueIfNeeded(ByteBuffer value,
 int schemaId,
 VeniceCompressor compressor)
                                          throws IOException

For chunked records, the chunk assembler already decompresses the assembled value via RawBytesChunkingAdapter's decompressingInputStreamDecoder. For non-chunked records, the value is still compressed and needs explicit decompression here.

Throws:

IOException