Package com.linkedin.venice.pubsub.adapter.kafka.producer

Class ApacheKafkaProducerConfig

java.lang.Object

com.linkedin.venice.pubsub.adapter.kafka.producer.ApacheKafkaProducerConfig

public class ApacheKafkaProducerConfig
extends Object

This class holds all properties used to construct ApacheKafkaProducerAdapter (This class could be refactored to hold consumer properties as well). Tune and adjust the configs in this class to control the behavior of Apache Kafka producer.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	DEFAULT_KAFKA_BATCH_SIZE	Default Kafka batch size and linger time for better producer performance during ingestion.
static final String	DEFAULT_KAFKA_LINGER_MS
static final String	KAFKA_BOOTSTRAP_SERVERS
static final String	KAFKA_BUFFER_MEMORY
static final String	KAFKA_CLIENT_ID
static final String	KAFKA_COMPRESSION_TYPE
static final String	KAFKA_CONFIG_PREFIX	Legacy Kafka configs are using only kafka prefix.
static final String	KAFKA_KEY_SERIALIZER
static final String	KAFKA_LINGER_MS
static final String	KAFKA_OVER_SSL
static final String	KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS
static final String	KAFKA_PRODUCER_REQUEST_TIMEOUT_MS
static final String	KAFKA_PRODUCER_RETRIES_CONFIG
static final String	KAFKA_VALUE_SERIALIZER
static final String	PUBSUB_KAFKA_CLIENT_CONFIG_PREFIX
static final String	SSL_KAFKA_BOOTSTRAP_SERVERS
static final String	SSL_TO_KAFKA_LEGACY	Deprecated. Use KAFKA_OVER_SSL instead

Constructor Summary

Constructors

Constructor	Description
ApacheKafkaProducerConfig(VeniceProperties allVeniceProperties, String brokerAddressToOverride, String producerName, boolean strictConfigs)
ApacheKafkaProducerConfig(Properties allVeniceProperties)

Method Summary

Modifier and Type	Method	Description
static void	copyKafkaSASLProperties(VeniceProperties configuration, Properties properties, boolean stripPrefix)
static void	copyKafkaSASLProperties(Properties configuration, Properties properties, boolean stripPrefix)
String	getBrokerAddress()
Properties	getProducerProperties()
static String	getPubsubBrokerAddress(VeniceProperties properties)
static Properties	getValidProducerProperties(Properties extractedProperties)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

KAFKA_CONFIG_PREFIX

public static final String KAFKA_CONFIG_PREFIX

Legacy Kafka configs are using only kafka prefix. But now we are using pubsub.kafka prefix for all Kafka configs.

See Also:

• Constant Field Values

PUBSUB_KAFKA_CLIENT_CONFIG_PREFIX

public static final String PUBSUB_KAFKA_CLIENT_CONFIG_PREFIX

See Also:

• Constant Field Values

KAFKA_BOOTSTRAP_SERVERS

public static final String KAFKA_BOOTSTRAP_SERVERS

See Also:

• Constant Field Values

KAFKA_PRODUCER_RETRIES_CONFIG

public static final String KAFKA_PRODUCER_RETRIES_CONFIG

See Also:

• Constant Field Values

KAFKA_LINGER_MS

public static final String KAFKA_LINGER_MS

See Also:

• Constant Field Values

KAFKA_BUFFER_MEMORY

public static final String KAFKA_BUFFER_MEMORY

See Also:

• Constant Field Values

KAFKA_CLIENT_ID

public static final String KAFKA_CLIENT_ID

See Also:

• Constant Field Values

KAFKA_KEY_SERIALIZER

public static final String KAFKA_KEY_SERIALIZER

See Also:

• Constant Field Values

KAFKA_COMPRESSION_TYPE

public static final String KAFKA_COMPRESSION_TYPE

See Also:

• Constant Field Values

KAFKA_VALUE_SERIALIZER

public static final String KAFKA_VALUE_SERIALIZER

See Also:

• Constant Field Values

KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS

public static final String KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS

See Also:

• Constant Field Values

KAFKA_PRODUCER_REQUEST_TIMEOUT_MS

public static final String KAFKA_PRODUCER_REQUEST_TIMEOUT_MS

See Also:

• Constant Field Values

SSL_KAFKA_BOOTSTRAP_SERVERS

public static final String SSL_KAFKA_BOOTSTRAP_SERVERS

See Also:

• Constant Field Values

SSL_TO_KAFKA_LEGACY

@Deprecated
public static final String SSL_TO_KAFKA_LEGACY

Deprecated.

Use KAFKA_OVER_SSL instead

N.B. do not attempt to change spelling, "kakfa", without carefully replacing all instances in use and some of them may be external to this repo

See Also:

• Constant Field Values

KAFKA_OVER_SSL

public static final String KAFKA_OVER_SSL

See Also:

• Constant Field Values

DEFAULT_KAFKA_BATCH_SIZE

public static final String DEFAULT_KAFKA_BATCH_SIZE

Default Kafka batch size and linger time for better producer performance during ingestion.

See Also:

• Constant Field Values

DEFAULT_KAFKA_LINGER_MS

public static final String DEFAULT_KAFKA_LINGER_MS

See Also:

• Constant Field Values

Constructor Details

ApacheKafkaProducerConfig

public ApacheKafkaProducerConfig(Properties allVeniceProperties)

ApacheKafkaProducerConfig

public ApacheKafkaProducerConfig(VeniceProperties allVeniceProperties,
 String brokerAddressToOverride,
 String producerName,
 boolean strictConfigs)

Method Details

getProducerProperties

public Properties getProducerProperties()

getPubsubBrokerAddress

public static String getPubsubBrokerAddress(VeniceProperties properties)

getBrokerAddress

public String getBrokerAddress()

getValidProducerProperties

public static Properties getValidProducerProperties(Properties extractedProperties)

copyKafkaSASLProperties

public static void copyKafkaSASLProperties(VeniceProperties configuration,
 Properties properties,
 boolean stripPrefix)

copyKafkaSASLProperties

public static void copyKafkaSASLProperties(Properties configuration,
 Properties properties,
 boolean stripPrefix)