Package com.linkedin.venice.pubsub.api

Class PubSubMessageDeserializer

java.lang.Object

com.linkedin.venice.pubsub.api.PubSubMessageDeserializer

public class PubSubMessageDeserializer
extends Object

The class for deserializing messages from the pubsub specific message format to PubSubMessage

Constructor Summary

Constructors

Constructor	Description
PubSubMessageDeserializer(KafkaValueSerializer valueSerializer, ObjectPool<KafkaMessageEnvelope> putEnvelopePool, ObjectPool<KafkaMessageEnvelope> updateEnvelopePool)
PubSubMessageDeserializer(KafkaValueSerializer valueSerializer, ObjectPool<KafkaMessageEnvelope> putEnvelopePool, ObjectPool<KafkaMessageEnvelope> updateEnvelopePool, boolean producerTimestampFallbackEnabled)

Method Summary

Modifier and Type	Method	Description
void	close()
static PubSubMessageDeserializer	createDefaultDeserializer()	Do not use the following default deserializer in production code as it does not support schema evolution properly.
static PubSubMessageDeserializer	createOptimizedDeserializer()	Do not use the following default deserializer in production code as it does not support schema evolution properly.
DefaultPubSubMessage	deserialize(PubSubTopicPartition topicPartition, byte[] keyBytes, byte[] valueBytes, PubSubMessageHeaders headers, PubSubPosition pubSubPosition, Long timestamp)	Deserialize a message from the pubsub specific message format to PubSubMessage.
KafkaValueSerializer	getValueSerializer()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PubSubMessageDeserializer

public PubSubMessageDeserializer(KafkaValueSerializer valueSerializer,
 ObjectPool<KafkaMessageEnvelope> putEnvelopePool,
 ObjectPool<KafkaMessageEnvelope> updateEnvelopePool)

PubSubMessageDeserializer

public PubSubMessageDeserializer(KafkaValueSerializer valueSerializer,
 ObjectPool<KafkaMessageEnvelope> putEnvelopePool,
 ObjectPool<KafkaMessageEnvelope> updateEnvelopePool,
 boolean producerTimestampFallbackEnabled)

Method Details

deserialize

public DefaultPubSubMessage deserialize(PubSubTopicPartition topicPartition,
 byte[] keyBytes,
 byte[] valueBytes,
 PubSubMessageHeaders headers,
 PubSubPosition pubSubPosition,
 Long timestamp)

Deserialize a message from the pubsub specific message format to PubSubMessage.

Side effect on headers: if the input contains PubSubMessageHeaders.VENICE_TRANSPORT_PROTOCOL_HEADER, this method strips it via PubSubMessageHeaders.stripProtocolSchemaHeader(com.linkedin.venice.pubsub.api.PubSubMessageHeaders) — best-effort in-place when the headers object is mutable, copy-fallback otherwise. The header carries the entire ~16 KB Avro JSON for KafkaMessageEnvelope and is dead weight once the value envelope is built; pinning it per queued record has been observed to cost upwards of 10 GB on the DaVinci buffer queue during back-pressure. Callers that need to retain vtp after this call must pass a copy. Callers whose headers reference is shared across reads should pre-strip via PubSubMessageHeaders.stripProtocolSchemaHeaderCopy(com.linkedin.venice.pubsub.api.PubSubMessageHeaders) themselves.

Parameters:

topicPartition - the topic partition from which the message was read

keyBytes - the key bytes of the message

valueBytes - the value bytes of the message

headers - the headers of the message; may be modified in place to remove PubSubMessageHeaders.VENICE_TRANSPORT_PROTOCOL_HEADER

pubSubPosition - the position of the message in the topic partition

timestamp - the timestamp of the message

Returns:

the deserialized PubSubMessage

close

public void close()

getValueSerializer

public KafkaValueSerializer getValueSerializer()

createDefaultDeserializer

public static PubSubMessageDeserializer createDefaultDeserializer()

Do not use the following default deserializer in production code as it does not support schema evolution properly. It is only provided for convenience in test code.

createOptimizedDeserializer

public static PubSubMessageDeserializer createOptimizedDeserializer()

Do not use the following default deserializer in production code as it does not support schema evolution properly. It is only provided for convenience in test code.