Package com.linkedin.venice.pubsub.adapter.kafka.producer

Class ApacheKafkaProducerConfig

java.lang.Object

com.linkedin.venice.pubsub.adapter.kafka.producer.ApacheKafkaProducerConfig

public class ApacheKafkaProducerConfig
extends java.lang.Object

This class holds all properties used to construct ApacheKafkaProducerAdapter (This class could be refactored to hold consumer properties as well). Tune and adjust the configs in this class to control the behavior of Apache Kafka producer.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	DEFAULT_KAFKA_BATCH_SIZE	Default Kafka batch size and linger time for better producer performance during ingestion.
static java.lang.String	DEFAULT_KAFKA_LINGER_MS
static java.lang.String	KAFKA_BOOTSTRAP_SERVERS
static java.lang.String	KAFKA_BUFFER_MEMORY
static java.lang.String	KAFKA_CLIENT_ID
static java.lang.String	KAFKA_CONFIG_PREFIX
static java.lang.String	KAFKA_KEY_SERIALIZER
static java.lang.String	KAFKA_LINGER_MS
static java.lang.String	KAFKA_OVER_SSL
static java.lang.String	KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS
static java.lang.String	KAFKA_PRODUCER_REQUEST_TIMEOUT_MS
static java.lang.String	KAFKA_PRODUCER_RETRIES_CONFIG
static java.lang.String	KAFKA_VALUE_SERIALIZER
static java.lang.String	SSL_KAFKA_BOOTSTRAP_SERVERS
static java.lang.String	SSL_TO_KAFKA_LEGACY	Deprecated. Use KAFKA_OVER_SSL instead

Constructor Summary

Constructors

Constructor	Description
ApacheKafkaProducerConfig(VeniceProperties allVeniceProperties, java.lang.String brokerAddressToOverride, java.lang.String producerName, boolean strictConfigs)
ApacheKafkaProducerConfig(java.util.Properties allVeniceProperties)

Method Summary

Modifier and Type	Method	Description
static void	copyKafkaSASLProperties(VeniceProperties configuration, java.util.Properties properties, boolean stripPrefix)
static void	copyKafkaSASLProperties(java.util.Properties configuration, java.util.Properties properties, boolean stripPrefix)
java.lang.String	getBrokerAddress()
java.util.Properties	getProducerProperties()
static java.lang.String	getPubsubBrokerAddress(VeniceProperties properties)
static java.util.Properties	getValidProducerProperties(java.util.Properties extractedProperties)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

KAFKA_CONFIG_PREFIX

public static final java.lang.String KAFKA_CONFIG_PREFIX

See Also:

Constant Field Values

KAFKA_BOOTSTRAP_SERVERS

public static final java.lang.String KAFKA_BOOTSTRAP_SERVERS

See Also:

Constant Field Values

KAFKA_PRODUCER_RETRIES_CONFIG

public static final java.lang.String KAFKA_PRODUCER_RETRIES_CONFIG

See Also:

Constant Field Values

KAFKA_LINGER_MS

public static final java.lang.String KAFKA_LINGER_MS

See Also:

Constant Field Values

KAFKA_BUFFER_MEMORY

public static final java.lang.String KAFKA_BUFFER_MEMORY

See Also:

Constant Field Values

KAFKA_CLIENT_ID

public static final java.lang.String KAFKA_CLIENT_ID

See Also:

Constant Field Values

KAFKA_KEY_SERIALIZER

public static final java.lang.String KAFKA_KEY_SERIALIZER

See Also:

Constant Field Values

KAFKA_VALUE_SERIALIZER

public static final java.lang.String KAFKA_VALUE_SERIALIZER

See Also:

Constant Field Values

KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS

public static final java.lang.String KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS

See Also:

Constant Field Values

KAFKA_PRODUCER_REQUEST_TIMEOUT_MS

public static final java.lang.String KAFKA_PRODUCER_REQUEST_TIMEOUT_MS

See Also:

Constant Field Values

SSL_KAFKA_BOOTSTRAP_SERVERS

public static final java.lang.String SSL_KAFKA_BOOTSTRAP_SERVERS

See Also:

Constant Field Values

SSL_TO_KAFKA_LEGACY

@Deprecated
public static final java.lang.String SSL_TO_KAFKA_LEGACY

Deprecated.

Use KAFKA_OVER_SSL instead

N.B. do not attempt to change spelling, "kakfa", without carefully replacing all instances in use and some of them may be external to this repo

See Also:

Constant Field Values

KAFKA_OVER_SSL

public static final java.lang.String KAFKA_OVER_SSL

See Also:

Constant Field Values

DEFAULT_KAFKA_BATCH_SIZE

public static final java.lang.String DEFAULT_KAFKA_BATCH_SIZE

Default Kafka batch size and linger time for better producer performance during ingestion.

See Also:

Constant Field Values

DEFAULT_KAFKA_LINGER_MS

public static final java.lang.String DEFAULT_KAFKA_LINGER_MS

See Also:

Constant Field Values

Constructor Detail

ApacheKafkaProducerConfig

public ApacheKafkaProducerConfig(java.util.Properties allVeniceProperties)

ApacheKafkaProducerConfig

public ApacheKafkaProducerConfig(VeniceProperties allVeniceProperties,
                                 java.lang.String brokerAddressToOverride,
                                 java.lang.String producerName,
                                 boolean strictConfigs)

Method Detail

getProducerProperties

public java.util.Properties getProducerProperties()

getPubsubBrokerAddress

public static java.lang.String getPubsubBrokerAddress(VeniceProperties properties)

getBrokerAddress

public java.lang.String getBrokerAddress()

getValidProducerProperties

public static java.util.Properties getValidProducerProperties(java.util.Properties extractedProperties)

copyKafkaSASLProperties

public static void copyKafkaSASLProperties(VeniceProperties configuration,
                                           java.util.Properties properties,
                                           boolean stripPrefix)

copyKafkaSASLProperties

public static void copyKafkaSASLProperties(java.util.Properties configuration,
                                           java.util.Properties properties,
                                           boolean stripPrefix)