Package com.linkedin.venice.compression
Class ZstdWithDictCompressor
- java.lang.Object
-
- com.linkedin.venice.compression.VeniceCompressor
-
- com.linkedin.venice.compression.ZstdWithDictCompressor
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
public class ZstdWithDictCompressor extends VeniceCompressor
-
-
Field Summary
-
Fields inherited from class com.linkedin.venice.compression.VeniceCompressor
SCHEMA_HEADER_LENGTH
-
-
Constructor Summary
Constructors Constructor Description ZstdWithDictCompressor(byte[] dictionary, int level)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static byte[]
buildDictionaryOnSyntheticAvroData()
Build a dictionary based on synthetic data.void
close()
byte[]
compress(byte[] data)
java.nio.ByteBuffer
compress(java.nio.ByteBuffer data, int startPositionOfOutput)
java.nio.ByteBuffer
decompress(byte[] data, int offset, int length)
java.io.InputStream
decompress(java.io.InputStream inputStream)
java.nio.ByteBuffer
decompress(java.nio.ByteBuffer data)
java.nio.ByteBuffer
decompressAndPrependSchemaHeader(byte[] data, int offset, int length, int schemaHeader)
This method tries to decompress data and maybe prepend the schema header.boolean
equals(java.lang.Object o)
int
hashCode()
-
Methods inherited from class com.linkedin.venice.compression.VeniceCompressor
getCompressionStrategy
-
-
-
-
Method Detail
-
compress
public byte[] compress(byte[] data)
- Specified by:
compress
in classVeniceCompressor
-
compress
public java.nio.ByteBuffer compress(java.nio.ByteBuffer data, int startPositionOfOutput) throws java.io.IOException
- Specified by:
compress
in classVeniceCompressor
- Throws:
java.io.IOException
-
decompress
public java.nio.ByteBuffer decompress(java.nio.ByteBuffer data) throws java.io.IOException
- Specified by:
decompress
in classVeniceCompressor
- Throws:
java.io.IOException
-
decompress
public java.nio.ByteBuffer decompress(byte[] data, int offset, int length) throws java.io.IOException
- Specified by:
decompress
in classVeniceCompressor
- Throws:
java.io.IOException
-
decompressAndPrependSchemaHeader
public java.nio.ByteBuffer decompressAndPrependSchemaHeader(byte[] data, int offset, int length, int schemaHeader) throws java.io.IOException
Description copied from class:VeniceCompressor
This method tries to decompress data and maybe prepend the schema header. The returned ByteBuffer will be backed by byte array that starts with schema header, followed by the decompressed data. The ByteBuffer will be positioned at the beginning of the decompressed data and the remaining of the ByteBuffer will be the length of the decompressed data.- Specified by:
decompressAndPrependSchemaHeader
in classVeniceCompressor
- Throws:
java.io.IOException
-
decompress
public java.io.InputStream decompress(java.io.InputStream inputStream) throws java.io.IOException
- Specified by:
decompress
in classVeniceCompressor
- Throws:
java.io.IOException
-
close
public void close() throws java.io.IOException
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Overrides:
close
in classVeniceCompressor
- Throws:
java.io.IOException
-
buildDictionaryOnSyntheticAvroData
public static byte[] buildDictionaryOnSyntheticAvroData()
Build a dictionary based on synthetic data. Used for empty push where there is no available dictionary to retrieve from a push job.- Returns:
- a zstd compression dictionary trained on small amount of avro data
-
hashCode
public int hashCode()
- Overrides:
hashCode
in classjava.lang.Object
-
equals
public boolean equals(java.lang.Object o)
- Overrides:
equals
in classjava.lang.Object
-
-