Package com.linkedin.venice.pubsub.adapter.kafka.producer

Class ApacheKafkaProducerAdapter

java.lang.Object

com.linkedin.venice.pubsub.adapter.kafka.producer.ApacheKafkaProducerAdapter

All Implemented Interfaces:

PubSubProducerAdapter

public class ApacheKafkaProducerAdapter
extends Object
implements PubSubProducerAdapter

A wrapper over Apache Kafka producer which implements PubSubProducerAdapter

Constructor Summary

Constructors

Constructor	Description
ApacheKafkaProducerAdapter(ApacheKafkaProducerConfig producerConfig)

Method Summary

Modifier and Type	Method	Description
void	close(long closeTimeOutMs)
void	flush()
String	getBrokerAddress()
it.unimi.dsi.fastutil.objects.Object2DoubleMap<String>	getMeasurableProducerMetrics()
int	getNumberOfPartitions(String topic)	Deprecated.
CompletableFuture<PubSubProduceResult>	sendMessage(PubSubTopicPartition pubSubTopicPartition, byte[] keyBytes, byte[] valueBytes, PubSubMessageHeaders pubsubMessageHeaders, PubSubProducerCallback pubsubProducerCallback)	Sends a message to a PubSub topic asynchronously and returns a CompletableFuture representing the result of the produce operation.
CompletableFuture<PubSubProduceResult>	sendMessage(String topic, Integer partition, KafkaKey key, KafkaMessageEnvelope value, PubSubMessageHeaders pubsubMessageHeaders, PubSubProducerCallback pubsubProducerCallback)	Sends a message to the Kafka Producer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ApacheKafkaProducerAdapter

public ApacheKafkaProducerAdapter(ApacheKafkaProducerConfig producerConfig)

Parameters:

producerConfig - contains producer configs

Method Details

getNumberOfPartitions

@Deprecated
public int getNumberOfPartitions(String topic)

Deprecated.

N.B.: This is an expensive call, the result of which should be cached.

Specified by:

getNumberOfPartitions in interface PubSubProducerAdapter

Parameters:

topic - for which we want to request the number of partitions.

Returns:

the number of partitions for this topic.

sendMessage

public CompletableFuture<PubSubProduceResult> sendMessage(String topic,
 Integer partition,
 KafkaKey key,
 KafkaMessageEnvelope value,
 PubSubMessageHeaders pubsubMessageHeaders,
 PubSubProducerCallback pubsubProducerCallback)

Sends a message to the Kafka Producer. If everything is set up correctly, it will show up in Kafka log.

Specified by:

sendMessage in interface PubSubProducerAdapter

Parameters:

topic - - The topic to be sent to.

key - - The key of the message to be sent.

value - - The KafkaMessageEnvelope, which acts as the Kafka value.

pubsubProducerCallback - - The callback function, which will be triggered when Kafka client sends out the message.

partition - The partition to which the message should be sent.

pubsubMessageHeaders - Additional headers to be included with the message.

Returns:

- A Future of PubSubProduceResult, which will be completed when the message is sent to pub-sub server successfully.

Throws:

PubSubOpTimeoutException - - If the operation times out.

PubSubTopicAuthorizationException - - If the producer is not authorized to send to the topic.

PubSubTopicDoesNotExistException - - If the topic does not exist.

PubSubClientRetriableException - - If the operation fails due to transient reasons.

PubSubClientException - - If the operation fails due to other reasons.

sendMessage

public CompletableFuture<PubSubProduceResult> sendMessage(PubSubTopicPartition pubSubTopicPartition,
 byte[] keyBytes,
 byte[] valueBytes,
 PubSubMessageHeaders pubsubMessageHeaders,
 PubSubProducerCallback pubsubProducerCallback)

Description copied from interface: PubSubProducerAdapter

Sends a message to a PubSub topic asynchronously and returns a CompletableFuture representing the result of the produce operation.

Specified by:

sendMessage in interface PubSubProducerAdapter

Parameters:

pubSubTopicPartition - The partition of the PubSub topic to which the message will be sent.

keyBytes - The key associated with the message, used for partitioning and message retrieval, represented as a byte array.

valueBytes - The message payload to be sent to the PubSub topic, represented as a byte array.

pubsubMessageHeaders - Additional headers to be included with the message, encapsulated in a PubSubMessageHeaders object.

pubsubProducerCallback - An optional callback to handle the result of the produce operation, encapsulated in a PubSubProducerCallback object.

Returns:

A CompletableFuture representing the result of the produce operation, encapsulated in a PubSubProduceResult object.

flush

public void flush()

Specified by:

flush in interface PubSubProducerAdapter

close

public void close(long closeTimeOutMs)

Specified by:

close in interface PubSubProducerAdapter

getMeasurableProducerMetrics

public it.unimi.dsi.fastutil.objects.Object2DoubleMap<String> getMeasurableProducerMetrics()

Specified by:

getMeasurableProducerMetrics in interface PubSubProducerAdapter

getBrokerAddress

public String getBrokerAddress()

Specified by:

getBrokerAddress in interface PubSubProducerAdapter