Package com.linkedin.venice.serialization.avro

Class InternalAvroSpecificSerializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>

java.lang.Object
com.linkedin.venice.serialization.avro.InternalAvroSpecificSerializer<SPECIFIC_RECORD>
All Implemented Interfaces:
VeniceKafkaSerializer<SPECIFIC_RECORD>, Closeable, AutoCloseable, org.apache.kafka.common.serialization.Deserializer<SPECIFIC_RECORD>, org.apache.kafka.common.serialization.Serializer<SPECIFIC_RECORD>
Direct Known Subclasses:
ChunkedKeySuffixSerializer, ChunkedValueManifestSerializer, KafkaValueSerializer
public class InternalAvroSpecificSerializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord> extends Object implements VeniceKafkaSerializer<SPECIFIC_RECORD>
Serializer for translating a versioned protocol of Avro records. The protocol is the following: 1st byte: The magic byte, should always equal 'magicByte'. 2nd byte: The protocol version 3rd byte and onward: The payload (a single binary-encoded Avro record) encoded with a writer schema determined by the protocol version specified in #2.

  • Field Details

    • VENICE_SCHEMA_READER_CONFIG

      @Deprecated public static final String VENICE_SCHEMA_READER_CONFIG
      Deprecated.
      Used to configure the schemaReader. Deprecated: This path has now been superseded by setSchemaReader(SchemaReader), which is used everywhere except in com.linkedin.venice.kafka.KafkaClientFactory#getConsumer(Properties, PubSubMessageDeserializer) ()}. Once that usage is also eliminated we could remove the config from here.
      See Also:

    • MAX_ATTEMPTS_FOR_SCHEMA_READER

      public static final int MAX_ATTEMPTS_FOR_SCHEMA_READER
      See Also:

    • WAIT_TIME_BETWEEN_SCHEMA_READER_ATTEMPTS_IN_MS

      public static final int WAIT_TIME_BETWEEN_SCHEMA_READER_ATTEMPTS_IN_MS
      See Also:

    • SENTINEL_PROTOCOL_VERSION_USED_FOR_UNDETECTABLE_COMPILED_SCHEMA

      public static final int SENTINEL_PROTOCOL_VERSION_USED_FOR_UNDETECTABLE_COMPILED_SCHEMA
      See Also:

    • SENTINEL_PROTOCOL_VERSION_USED_FOR_UNVERSIONED_PROTOCOL

      public static final int SENTINEL_PROTOCOL_VERSION_USED_FOR_UNVERSIONED_PROTOCOL
      See Also:

  • Constructor Details

  • Method Details

    • close

      public void close()
      Close this serializer. This method has to be idempotent if the serializer is used in KafkaProducer because it might be called multiple times.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Specified by:
      close in interface org.apache.kafka.common.serialization.Deserializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Specified by:
      close in interface org.apache.kafka.common.serialization.Serializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Specified by:
      close in interface VeniceKafkaSerializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>

    • knownProtocols

      public it.unimi.dsi.fastutil.ints.IntSet knownProtocols()

    • getCompiledProtocol

      public org.apache.avro.Schema getCompiledProtocol()

    • configure

      public void configure(Map<String,?> configMap, boolean isKey)
      Configure this class. KafkaConsumer(Properties) would eventually call configure(Map, boolean) which would pass in the customized Kafka config map with schema reader.
      Specified by:
      configure in interface org.apache.kafka.common.serialization.Deserializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Specified by:
      configure in interface org.apache.kafka.common.serialization.Serializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Specified by:
      configure in interface VeniceKafkaSerializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Parameters:
      configMap - configs in key/value pairs
      isKey - whether is for key or value

    • setSchemaReader

      public void setSchemaReader(SchemaReader schemaReader)

    • serialize

      public byte[] serialize(String topic, SPECIFIC_RECORD object)
      Construct an array of bytes from the given object
      Specified by:
      serialize in interface org.apache.kafka.common.serialization.Serializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Specified by:
      serialize in interface VeniceKafkaSerializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Parameters:
      topic - Topic to which the object belongs (for API compatibility reason only, but unused)
      object - A InternalAvroSpecificSerializer instance to be serialized.
      Returns:
      The Avro binary format bytes which represent the

    • serialize

      public ByteBuffer serialize(SPECIFIC_RECORD object)

    • deserialize

      public SPECIFIC_RECORD deserialize(String topic, byte[] bytes)
      Create an object from an array of bytes This method is used by the Kafka consumer. These calls are always intended to be for protocols which use a magic byte and a protocol version, both of which are stored in the header, before the payload.
      Specified by:
      deserialize in interface org.apache.kafka.common.serialization.Deserializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Specified by:
      deserialize in interface VeniceKafkaSerializer<SPECIFIC_RECORD extends org.apache.avro.specific.SpecificRecord>
      Parameters:
      topic - Topic to which the array of bytes belongs (only there to implement the interface, but otherwise useless)
      bytes - An array of bytes representing the object's data serialized in Avro binary format.
      Returns:
      A InternalAvroSpecificSerializer deserialized from the bytes

    • deserialize

      public SPECIFIC_RECORD deserialize(byte[] bytes, SPECIFIC_RECORD reuse)

    • deserialize

      public SPECIFIC_RECORD deserialize(byte[] bytes, int protocolVersion)

    • deserialize

      public SPECIFIC_RECORD deserialize(byte[] bytes, int protocolVersion, SPECIFIC_RECORD reuse)

    • deserialize

      public SPECIFIC_RECORD deserialize(byte[] bytes, org.apache.avro.Schema providedProtocolSchema, SPECIFIC_RECORD reuse)

    • createBinaryDecoder

      protected org.apache.avro.io.BinaryDecoder createBinaryDecoder(byte[] bytes, int offset, int length, org.apache.avro.io.BinaryDecoder reuse)

    • removeAllSchemas

      public void removeAllSchemas()