Package com.linkedin.venice.pubsub.adapter.kafka.admin

Class ApacheKafkaAdminAdapter

java.lang.Object

com.linkedin.venice.pubsub.adapter.kafka.admin.ApacheKafkaAdminAdapter

All Implemented Interfaces:

PubSubAdminAdapter, Closeable, AutoCloseable

public class ApacheKafkaAdminAdapter
extends Object
implements PubSubAdminAdapter

An implementation of PubSubAdminAdapter for Apache Kafka.

Constructor Summary

Constructors

Constructor	Description
ApacheKafkaAdminAdapter(ApacheKafkaAdminConfig apacheKafkaAdminConfig, PubSubTopicRepository pubSubTopicRepository)

Method Summary

Modifier and Type	Method	Description
void	close()
boolean	containsTopic(PubSubTopic pubSubTopic)	Checks if a Kafka topic exists.
boolean	containsTopicWithPartitionCheck(PubSubTopicPartition pubSubTopicPartition)	Checks if a topic exists and has the given partition
void	createTopic(PubSubTopic pubSubTopic, int numPartitions, int replicationFactor, PubSubTopicConfiguration pubSubTopicConfiguration)	Creates a new topic in the PubSub system with the given parameters.
void	deleteTopic(PubSubTopic pubSubTopic, Duration timeout)	Delete a given topic.
Map<PubSubTopic,Long>	getAllTopicRetentions()	Retrieves the retention settings for all Kafka topics.
String	getClassName()
Map<PubSubTopic,PubSubTopicConfiguration>	getSomeTopicConfigs(Set<PubSubTopic> pubSubTopics)	Retrieves the configurations for a set of Kafka topics.
PubSubTopicConfiguration	getTopicConfig(PubSubTopic pubSubTopic)	Retrieves the configuration of a Kafka topic.
long	getTopicConfigMaxRetryInMs()
Set<PubSubTopic>	listAllTopics()	Retrieves a set of all available PubSub topics from the Kafka cluster.
void	setTopicConfig(PubSubTopic pubSubTopic, PubSubTopicConfiguration pubSubTopicConfiguration)	Sets the configuration for a Kafka topic.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.venice.pubsub.api.PubSubAdminAdapter

containsTopicWithExpectationAndRetry, containsTopicWithExpectationAndRetry, containsTopicWithPartitionCheckExpectationAndRetry, containsTopicWithPartitionCheckExpectationAndRetry, getRetriableExceptions, getTopicConfigWithRetry

Constructor Details

ApacheKafkaAdminAdapter

public ApacheKafkaAdminAdapter(ApacheKafkaAdminConfig apacheKafkaAdminConfig,
 PubSubTopicRepository pubSubTopicRepository)

Method Details

createTopic

public void createTopic(PubSubTopic pubSubTopic,
 int numPartitions,
 int replicationFactor,
 PubSubTopicConfiguration pubSubTopicConfiguration)

Creates a new topic in the PubSub system with the given parameters.

Specified by:

createTopic in interface PubSubAdminAdapter

Parameters:

pubSubTopic - The topic to be created.

numPartitions - The number of partitions to be created for the topic.

replicationFactor - The number of replicas for each partition.

pubSubTopicConfiguration - Additional topic configuration such as retention, compaction policy, etc.

Throws:

IllegalArgumentException - If the replication factor is invalid.

PubSubTopicExistsException - If a topic with the same name already exists.

PubSubClientRetriableException - If the operation failed due to a retriable error.

PubSubClientException - For all other issues related to the PubSub client.

deleteTopic

public void deleteTopic(PubSubTopic pubSubTopic,
 Duration timeout)

Delete a given topic. The calling thread will block until the topic is deleted or the timeout is reached.

Specified by:

deleteTopic in interface PubSubAdminAdapter

Parameters:

pubSubTopic - The topic to delete.

timeout - The maximum duration to wait for the deletion to complete.

Throws:

PubSubTopicDoesNotExistException - If the topic does not exist.

PubSubOpTimeoutException - If the operation times out.

PubSubClientRetriableException - If the operation fails and can be retried.

PubSubClientException - For all other issues related to the PubSub client.

getTopicConfig

public PubSubTopicConfiguration getTopicConfig(PubSubTopic pubSubTopic)

Retrieves the configuration of a Kafka topic.

Specified by:

getTopicConfig in interface PubSubAdminAdapter

Parameters:

pubSubTopic - The PubSubTopic representing the Kafka topic for which to retrieve the configuration.

Returns:

The configuration of the specified Kafka topic as a PubSubTopicConfiguration object.

Throws:

PubSubTopicDoesNotExistException - If the specified Kafka topic does not exist.

PubSubClientRetriableException - If a retriable error occurs while attempting to retrieve the configuration.

PubSubClientException - If an error occurs while attempting to retrieve the configuration or if the current thread is interrupted while attempting to retrieve the configuration.

listAllTopics

public Set<PubSubTopic> listAllTopics()

Retrieves a set of all available PubSub topics from the Kafka cluster.

Specified by:

listAllTopics in interface PubSubAdminAdapter

Returns:

A Set of PubSubTopic objects representing all available Kafka topics.

Throws:

PubSubClientRetriableException - If a retriable error occurs while attempting to retrieve the list of topics.

PubSubClientException - If an error occurs while attempting to retrieve the list of topics or the current thread is interrupted while attempting to retrieve the list of topics.

setTopicConfig

public void setTopicConfig(PubSubTopic pubSubTopic,
 PubSubTopicConfiguration pubSubTopicConfiguration)
                    throws PubSubTopicDoesNotExistException

Sets the configuration for a Kafka topic.

Specified by:

setTopicConfig in interface PubSubAdminAdapter

Parameters:

pubSubTopic - The PubSubTopic for which to set the configuration.

pubSubTopicConfiguration - The configuration to be set for the specified Kafka topic.

Throws:

PubSubTopicDoesNotExistException - If the specified Kafka topic does not exist.

PubSubClientException - If an error occurs while attempting to set the topic configuration or if the current thread is interrupted while attempting to set the topic configuration.

containsTopic

public boolean containsTopic(PubSubTopic pubSubTopic)

Checks if a Kafka topic exists.

Specified by:

containsTopic in interface PubSubAdminAdapter

Parameters:

pubSubTopic - The PubSubTopic to check for existence.

Returns:

true if the specified Kafka topic exists, false otherwise.

Throws:

PubSubClientRetriableException - If a retriable error occurs while attempting to check topic existence.

PubSubClientException - If an error occurs while attempting to check topic existence.

containsTopicWithPartitionCheck

public boolean containsTopicWithPartitionCheck(PubSubTopicPartition pubSubTopicPartition)

Checks if a topic exists and has the given partition

Specified by:

containsTopicWithPartitionCheck in interface PubSubAdminAdapter

Parameters:

pubSubTopicPartition - The PubSubTopicPartition representing the Kafka topic and partition to check.

Returns:

true if the specified Kafka topic partition exists, false otherwise.

Throws:

PubSubClientRetriableException - If a retriable error occurs while attempting to check partition existence.

PubSubClientException - If an error occurs while attempting to check partition existence or of the current thread is interrupted while attempting to check partition existence.

getAllTopicRetentions

public Map<PubSubTopic,Long> getAllTopicRetentions()

Retrieves the retention settings for all Kafka topics.

Specified by:

getAllTopicRetentions in interface PubSubAdminAdapter

Returns:

A map of Kafka topics and their corresponding retention settings in milliseconds. If a topic does not have a retention setting, it will be mapped to PubSubConstants.PUBSUB_TOPIC_UNKNOWN_RETENTION.

Throws:

PubSubClientRetriableException - If a retriable error occurs while attempting to retrieve retention settings.

PubSubClientException - If an error occurs while attempting to retrieve retention settings or if the current thread is interrupted while attempting to retrieve retention settings.

getSomeTopicConfigs

public Map<PubSubTopic,PubSubTopicConfiguration> getSomeTopicConfigs(Set<PubSubTopic> pubSubTopics)

Retrieves the configurations for a set of Kafka topics.

Specified by:

getSomeTopicConfigs in interface PubSubAdminAdapter

Parameters:

pubSubTopics - The set of Kafka topics to retrieve configurations for.

Returns:

A map of Kafka topics and their corresponding configurations.

Throws:

PubSubClientRetriableException - If a retriable error occurs while attempting to retrieve configurations.

PubSubClientException - If an error occurs while attempting to retrieve configurations or if the current thread is interrupted while attempting to retrieve configurations.

getClassName

public String getClassName()

Specified by:

getClassName in interface PubSubAdminAdapter

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

getTopicConfigMaxRetryInMs

public long getTopicConfigMaxRetryInMs()

Specified by:

getTopicConfigMaxRetryInMs in interface PubSubAdminAdapter