Package com.linkedin.venice.compression

Class ZstdWithDictCompressor

java.lang.Object

com.linkedin.venice.compression.VeniceCompressor

com.linkedin.venice.compression.ZstdWithDictCompressor

All Implemented Interfaces:

Closeable, AutoCloseable

public class ZstdWithDictCompressor
extends VeniceCompressor

Field Summary

Fields inherited from class com.linkedin.venice.compression.VeniceCompressor

SCHEMA_HEADER_LENGTH

Constructor Summary

Constructors

Constructor	Description
ZstdWithDictCompressor(byte[] dictionary, int level)

Method Summary

Modifier and Type	Method	Description
static byte[]	buildDictionaryOnSyntheticAvroData()	Build a dictionary based on synthetic data.
protected void	closeInternal()
protected byte[]	compressInternal(byte[] data)
protected ByteBuffer	compressInternal(ByteBuffer data, int startPositionOfOutput)
protected ByteBuffer	decompressAndPrependSchemaHeaderInternal(byte[] data, int offset, int length, int schemaHeader)
protected ByteBuffer	decompressInternal(byte[] data, int offset, int length)
protected InputStream	decompressInternal(InputStream inputStream)
protected ByteBuffer	decompressInternal(ByteBuffer data)
boolean	equals(Object o)
int	hashCode()

Methods inherited from class com.linkedin.venice.compression.VeniceCompressor

close, compress, compress, decompress, decompress, decompress, decompressAndPrependSchemaHeader, getCompressionStrategy

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ZstdWithDictCompressor

public ZstdWithDictCompressor(byte[] dictionary,
 int level)

Method Details

compressInternal

protected byte[] compressInternal(byte[] data)

Specified by:

compressInternal in class VeniceCompressor

compressInternal

protected ByteBuffer compressInternal(ByteBuffer data,
 int startPositionOfOutput)
                               throws IOException

Specified by:

compressInternal in class VeniceCompressor

Throws:

IOException

decompressInternal

protected ByteBuffer decompressInternal(ByteBuffer data)
                                 throws IOException

Specified by:

decompressInternal in class VeniceCompressor

Throws:

IOException

decompressInternal

protected ByteBuffer decompressInternal(byte[] data,
 int offset,
 int length)
                                 throws IOException

Specified by:

decompressInternal in class VeniceCompressor

Throws:

IOException

decompressAndPrependSchemaHeaderInternal

protected ByteBuffer decompressAndPrependSchemaHeaderInternal(byte[] data,
 int offset,
 int length,
 int schemaHeader)
                                                       throws IOException

Specified by:

decompressAndPrependSchemaHeaderInternal in class VeniceCompressor

Throws:

IOException

decompressInternal

protected InputStream decompressInternal(InputStream inputStream)
                                  throws IOException

Specified by:

decompressInternal in class VeniceCompressor

Throws:

IOException

closeInternal

protected void closeInternal()
                      throws IOException

Specified by:

closeInternal in class VeniceCompressor

Throws:

IOException

buildDictionaryOnSyntheticAvroData

public static byte[] buildDictionaryOnSyntheticAvroData()

Build a dictionary based on synthetic data. Used for empty push where there is no available dictionary to retrieve from a push job.

Returns:

a zstd compression dictionary trained on small amount of avro data

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object o)

Overrides:

equals in class Object