Package com.linkedin.venice

Class ConfigKeys

java.lang.Object
com.linkedin.venice.ConfigKeys
public class ConfigKeys extends Object

  • Field Details

    • CLUSTER_NAME

      public static final String CLUSTER_NAME
      See Also:

    • ZOOKEEPER_ADDRESS

      public static final String ZOOKEEPER_ADDRESS
      See Also:

    • ADMIN_PORT

      public static final String ADMIN_PORT
      See Also:

    • ADMIN_HOSTNAME

      public static final String ADMIN_HOSTNAME
      See Also:

    • ADMIN_SECURE_PORT

      public static final String ADMIN_SECURE_PORT
      See Also:

    • ADMIN_CHECK_READ_METHOD_FOR_KAFKA

      public static final String ADMIN_CHECK_READ_METHOD_FOR_KAFKA
      Whether controller should check "Read" method against Kafka wildcard ACL while users request for a topic to write. By default, the config value should be true, but setting it to false would allow us to release new version of controller when the "Read" method check is not working as expected.
      See Also:

    • PERSISTENCE_TYPE

      public static final String PERSISTENCE_TYPE
      See Also:

    • KAFKA_CONFIG_PREFIX

      public static final String KAFKA_CONFIG_PREFIX
      See Also:

    • KAFKA_BOOTSTRAP_SERVERS

      public static final String KAFKA_BOOTSTRAP_SERVERS
      See Also:

    • SSL_KAFKA_BOOTSTRAP_SERVERS

      public static final String SSL_KAFKA_BOOTSTRAP_SERVERS
      See Also:

    • KAFKA_LINGER_MS

      public static final String KAFKA_LINGER_MS
      See Also:

    • KAFKA_PRODUCER_REQUEST_TIMEOUT_MS

      public static final String KAFKA_PRODUCER_REQUEST_TIMEOUT_MS
      See Also:

    • KAFKA_PRODUCER_RETRIES_CONFIG

      public static final String KAFKA_PRODUCER_RETRIES_CONFIG
      See Also:

    • KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS

      public static final String KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS
      See Also:

    • KAFKA_CLIENT_ID_CONFIG

      public static final String KAFKA_CLIENT_ID_CONFIG
      See Also:

    • KAFKA_GROUP_ID_CONFIG

      public static final String KAFKA_GROUP_ID_CONFIG
      See Also:

    • KAFKA_AUTO_OFFSET_RESET_CONFIG

      public static final String KAFKA_AUTO_OFFSET_RESET_CONFIG
      See Also:

    • KAFKA_ENABLE_AUTO_COMMIT_CONFIG

      public static final String KAFKA_ENABLE_AUTO_COMMIT_CONFIG
      See Also:

    • KAFKA_FETCH_MIN_BYTES_CONFIG

      public static final String KAFKA_FETCH_MIN_BYTES_CONFIG
      See Also:

    • KAFKA_FETCH_MAX_BYTES_CONFIG

      public static final String KAFKA_FETCH_MAX_BYTES_CONFIG
      See Also:

    • KAFKA_MAX_POLL_RECORDS_CONFIG

      public static final String KAFKA_MAX_POLL_RECORDS_CONFIG
      See Also:

    • KAFKA_FETCH_MAX_WAIT_MS_CONFIG

      public static final String KAFKA_FETCH_MAX_WAIT_MS_CONFIG
      See Also:

    • KAFKA_MAX_PARTITION_FETCH_BYTES_CONFIG

      public static final String KAFKA_MAX_PARTITION_FETCH_BYTES_CONFIG
      See Also:

    • KAFKA_ADMIN_GET_TOPIC_CONFIG_MAX_RETRY_TIME_SEC

      public static final String KAFKA_ADMIN_GET_TOPIC_CONFIG_MAX_RETRY_TIME_SEC
      See Also:

    • KAFKA_FETCH_QUOTA_TIME_WINDOW_MS

      public static final String KAFKA_FETCH_QUOTA_TIME_WINDOW_MS
      The time window used by the consumption throttler. Throttler will sum the requests during the time window and compare with the quota accumulated in the time window to see whether the usage exceeds quota or not.
      See Also:

    • KAFKA_FETCH_QUOTA_BYTES_PER_SECOND

      public static final String KAFKA_FETCH_QUOTA_BYTES_PER_SECOND
      See Also:

    • KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND

      public static final String KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND
      How many records that one server could consume from Kafka at most in one second. If the consume rate reached this quota, the consumption thread will be blocked until there is the available quota.
      See Also:

    • SERVER_KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND
      How many records that one server could consume from Kafka at most in one second from the specified regions. If the consume rate reached this quota, the consumption thread will be blocked until there is the available quota. The value for this config is read from cluster configs in Zk.
      See Also:

    • DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_SPEEDUP_ENABLED

      public static final String DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_SPEEDUP_ENABLED
      Whether to speed up DaVinci current version bootstrapping or not.
      See Also:

    • DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_QUOTA_RECORDS_PER_SECOND

      public static final String DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_QUOTA_RECORDS_PER_SECOND
      When DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_SPEEDUP_ENABLED is true, the following throttler will be applied when any current version is bootstrapping.
      See Also:

    • DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_QUOTA_BYTES_PER_SECOND

      public static final String DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_QUOTA_BYTES_PER_SECOND
      When DA_VINCI_CURRENT_VERSION_BOOTSTRAPPING_SPEEDUP_ENABLED is true, the following throttler will be applied when any current version is bootstrapping.
      See Also:

    • KAFKA_FETCH_QUOTA_UNORDERED_BYTES_PER_SECOND

      public static final String KAFKA_FETCH_QUOTA_UNORDERED_BYTES_PER_SECOND
      See Also:

    • KAFKA_FETCH_QUOTA_UNORDERED_RECORDS_PER_SECOND

      public static final String KAFKA_FETCH_QUOTA_UNORDERED_RECORDS_PER_SECOND
      See Also:

    • KAFKA_SECURITY_PROTOCOL

      public static final String KAFKA_SECURITY_PROTOCOL
      See Also:

    • PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_CONSUMER_POOL_SIZE

      public static final String PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_CONSUMER_POOL_SIZE
      Number of PubSub consumer clients to be used per topic manager for fetching metadata.
      See Also:

    • PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_THREAD_POOL_SIZE

      public static final String PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_THREAD_POOL_SIZE
      The size of the per topic manager thread pool for fetching metadata asynchronously.
      See Also:

    • SERVER_MAX_WAIT_AFTER_UNSUBSCRIBE_MS

      public static final String SERVER_MAX_WAIT_AFTER_UNSUBSCRIBE_MS
      During a state transition, it is unsafe to proceed without waiting for all inflight messages to be processed. This controls how long to wait for inflight messages after unsubscribing from a topic during a state transition.
      See Also:

    • CONTROLLER_NAME

      public static final String CONTROLLER_NAME
      See Also:

    • KAFKA_LOG_COMPACTION_FOR_HYBRID_STORES

      public static final String KAFKA_LOG_COMPACTION_FOR_HYBRID_STORES
      Whether to turn on Kafka's log compaction for the store-version topics of hybrid (and real-time only) stores. Will take effect at topic creation time, and when the hybrid config for the store is turned on.
      See Also:

    • KAFKA_MIN_IN_SYNC_REPLICAS

      public static final String KAFKA_MIN_IN_SYNC_REPLICAS
      The minimum number of in sync replicas to set for store version topics. Will use the Kafka cluster's default if not set.
      See Also:

    • KAFKA_MIN_IN_SYNC_REPLICAS_RT_TOPICS

      public static final String KAFKA_MIN_IN_SYNC_REPLICAS_RT_TOPICS
      The minimum number of in sync replicas to set for real-time buffer topics. Will use the Kafka cluster's default if not set.
      See Also:

    • KAFKA_MIN_IN_SYNC_REPLICAS_ADMIN_TOPICS

      public static final String KAFKA_MIN_IN_SYNC_REPLICAS_ADMIN_TOPICS
      The minimum number of in sync replicas to set for admin topics. Will use the Kafka cluster's default if not set.
      See Also:

    • KAFKA_REPLICATION_FACTOR

      public static final String KAFKA_REPLICATION_FACTOR
      The replication factor to set for store-version topics.
      See Also:

    • KAFKA_REPLICATION_FACTOR_RT_TOPICS

      public static final String KAFKA_REPLICATION_FACTOR_RT_TOPICS
      The replication factor to set for real-time buffer topics.
      See Also:

    • ENABLE_ACTIVE_ACTIVE_REPLICATION_AS_DEFAULT_FOR_HYBRID_STORE

      public static final String ENABLE_ACTIVE_ACTIVE_REPLICATION_AS_DEFAULT_FOR_HYBRID_STORE
      Cluster-level config to enable active-active replication for new hybrid stores.
      See Also:

    • ENABLE_BLOB_TRANSFER

      public static final String ENABLE_BLOB_TRANSFER
      Store-level config to enable blob transfers (feature flag)
      See Also:

    • CONTROLLER_SCHEMA_VALIDATION_ENABLED

      public static final String CONTROLLER_SCHEMA_VALIDATION_ENABLED
      Sets the default for whether to do schema validation or not for all stores
      See Also:

    • DEFAULT_READ_STRATEGY

      public static final String DEFAULT_READ_STRATEGY
      Fallback to remain compatible with the old config spelling. Ignored if "kafka.replication.factor" is present.
      See Also:

    • DEFAULT_OFFLINE_PUSH_STRATEGY

      public static final String DEFAULT_OFFLINE_PUSH_STRATEGY
      See Also:

    • DEFAULT_ROUTING_STRATEGY

      public static final String DEFAULT_ROUTING_STRATEGY
      See Also:

    • DEFAULT_REPLICA_FACTOR

      public static final String DEFAULT_REPLICA_FACTOR
      See Also:

    • DEFAULT_NUMBER_OF_PARTITION

      public static final String DEFAULT_NUMBER_OF_PARTITION
      See Also:

    • DEFAULT_NUMBER_OF_PARTITION_FOR_HYBRID

      public static final String DEFAULT_NUMBER_OF_PARTITION_FOR_HYBRID
      See Also:

    • DEFAULT_MAX_NUMBER_OF_PARTITIONS

      public static final String DEFAULT_MAX_NUMBER_OF_PARTITIONS
      See Also:

    • DEFAULT_PARTITION_SIZE

      public static final String DEFAULT_PARTITION_SIZE
      See Also:

    • ENABLE_PARTITION_COUNT_ROUND_UP

      public static final String ENABLE_PARTITION_COUNT_ROUND_UP
      Whether to round up the version-level partition count calculated by storage quota. Default is false.
      See Also:

    • PARTITION_COUNT_ROUND_UP_SIZE

      public static final String PARTITION_COUNT_ROUND_UP_SIZE
      If "enable.partition.count.round.up" is enabled, this config defines the round up size. Default is 1.
      See Also:

    • OFFLINE_JOB_START_TIMEOUT_MS

      public static final String OFFLINE_JOB_START_TIMEOUT_MS
      See Also:

    • DELAY_TO_REBALANCE_MS

      public static final String DELAY_TO_REBALANCE_MS
      See Also:

    • CLUSTER_TO_D2

      public static final String CLUSTER_TO_D2
      See Also:

    • CLUSTER_TO_SERVER_D2

      public static final String CLUSTER_TO_SERVER_D2
      See Also:

    • HELIX_SEND_MESSAGE_TIMEOUT_MS

      public static final String HELIX_SEND_MESSAGE_TIMEOUT_MS
      See Also:

    • REFRESH_ATTEMPTS_FOR_ZK_RECONNECT

      public static final String REFRESH_ATTEMPTS_FOR_ZK_RECONNECT
      See Also:

    • REFRESH_INTERVAL_FOR_ZK_RECONNECT_MS

      public static final String REFRESH_INTERVAL_FOR_ZK_RECONNECT_MS
      See Also:

    • KAFKA_READ_CYCLE_DELAY_MS

      public static final String KAFKA_READ_CYCLE_DELAY_MS
      See Also:

    • KAFKA_EMPTY_POLL_SLEEP_MS

      public static final String KAFKA_EMPTY_POLL_SLEEP_MS
      See Also:

    • KAFKA_FETCH_MIN_SIZE_PER_SEC

      public static final String KAFKA_FETCH_MIN_SIZE_PER_SEC
      See Also:

    • KAFKA_FETCH_MAX_SIZE_PER_SEC

      public static final String KAFKA_FETCH_MAX_SIZE_PER_SEC
      See Also:

    • KAFKA_FETCH_MAX_WAIT_TIME_MS

      public static final String KAFKA_FETCH_MAX_WAIT_TIME_MS
      See Also:

    • KAFKA_FETCH_PARTITION_MAX_SIZE_PER_SEC

      public static final String KAFKA_FETCH_PARTITION_MAX_SIZE_PER_SEC
      See Also:

    • CONTROLLER_CLUSTER_ZK_ADDRESSS

      public static final String CONTROLLER_CLUSTER_ZK_ADDRESSS
      See Also:

    • CONTROLLER_CLUSTER

      public static final String CONTROLLER_CLUSTER
      See Also:

    • CONTROLLER_RESOURCE_INSTANCE_GROUP_TAG

      public static final String CONTROLLER_RESOURCE_INSTANCE_GROUP_TAG
      See Also:

    • CONTROLLER_INSTANCE_TAG_LIST

      public static final String CONTROLLER_INSTANCE_TAG_LIST
      See Also:

    • CONTROLLER_GRPC_SERVER_ENABLED

      public static final String CONTROLLER_GRPC_SERVER_ENABLED
      Whether to enable gRPC server in controller or not.
      See Also:

    • CONTROLLER_ADMIN_GRPC_PORT

      public static final String CONTROLLER_ADMIN_GRPC_PORT
      A port for the controller to listen on for incoming requests. On this port, the controller will server non-ssl requests.
      See Also:

    • CONTROLLER_ADMIN_SECURE_GRPC_PORT

      public static final String CONTROLLER_ADMIN_SECURE_GRPC_PORT
      A port for the controller to listen on for incoming requests. On this port, the controller will only serve ssl requests.
      See Also:

    • CONTROLLER_GRPC_SERVER_THREAD_COUNT

      public static final String CONTROLLER_GRPC_SERVER_THREAD_COUNT
      Number of threads to use for the gRPC server in controller.
      See Also:

    • CONTROLLER_DISABLED_ROUTES

      public static final String CONTROLLER_DISABLED_ROUTES
      List of forbidden admin paths
      See Also:

    • DEPRECATED_TOPIC_RETENTION_MS

      public static final String DEPRECATED_TOPIC_RETENTION_MS
      The retention policy for deprecated topics, which includes topics for both failed jobs and retired store versions.
      See Also:

    • SERVICE_DISCOVERY_REGISTRATION_RETRY_MS

      public static final String SERVICE_DISCOVERY_REGISTRATION_RETRY_MS
      Time interval to retry registering service discovery announcers that failed to register. By default, this is set to 30000L ms.
      See Also:

    • FATAL_DATA_VALIDATION_FAILURE_TOPIC_RETENTION_MS

      public static final String FATAL_DATA_VALIDATION_FAILURE_TOPIC_RETENTION_MS
      See Also:

    • DEPRECATED_TOPIC_MAX_RETENTION_MS

      public static final String DEPRECATED_TOPIC_MAX_RETENTION_MS
      This config is to indicate the max retention policy we have setup for deprecated jobs currently and in the past. And this is used to decide whether the topic is deprecated or not during topic cleanup. The reason to have this config instead of using DEPRECATED_TOPIC_RETENTION_MS since the retention policy for deprecated jobs could change from time to time, and we need to use a max threshold to cover all the historical deprecated job topics.
      See Also:

    • TOPIC_CLEANUP_SLEEP_INTERVAL_BETWEEN_TOPIC_LIST_FETCH_MS

      public static final String TOPIC_CLEANUP_SLEEP_INTERVAL_BETWEEN_TOPIC_LIST_FETCH_MS
      Sleep interval between each topic list fetch from Kafka ZK in TopicCleanup service. We don't want to hit Kafka Zookeeper too frequently.
      See Also:

    • TOPIC_CLEANUP_DELAY_FACTOR

      public static final String TOPIC_CLEANUP_DELAY_FACTOR
      See Also:

    • CONTROLLER_BACKUP_VERSION_DEFAULT_RETENTION_MS

      public static final String CONTROLLER_BACKUP_VERSION_DEFAULT_RETENTION_MS
      The following config is to control the default retention time in milliseconds if it is not specified in store level.
      See Also:

    • CONTROLLER_BACKUP_VERSION_DELETION_SLEEP_MS

      public static final String CONTROLLER_BACKUP_VERSION_DELETION_SLEEP_MS
      See Also:

    • CONTROLLER_BACKUP_VERSION_RETENTION_BASED_CLEANUP_ENABLED

      public static final String CONTROLLER_BACKUP_VERSION_RETENTION_BASED_CLEANUP_ENABLED
      The following config is to control whether to enable backup version cleanup based on retention policy or not at cluster level.
      See Also:

    • CONTROLLER_BACKUP_VERSION_METADATA_FETCH_BASED_CLEANUP_ENABLED

      public static final String CONTROLLER_BACKUP_VERSION_METADATA_FETCH_BASED_CLEANUP_ENABLED
      The following config is to control whether to enable backup version cleanup based on router and server current version being served.
      See Also:

    • CONTROLLER_ZK_SHARED_META_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED

      public static final String CONTROLLER_ZK_SHARED_META_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED
      Whether to automatically create zk shared metadata system store in Controller or not
      See Also:

    • CONTROLLER_ENFORCE_SSL

      public static final String CONTROLLER_ENFORCE_SSL
      Whether controller should enforce SSL.
      See Also:

    • MULTI_REGION

      public static final String MULTI_REGION
      This config specifies if Venice is deployed in a multi-region mode
      See Also:

    • ADMIN_TOPIC_SOURCE_REGION

      public static final String ADMIN_TOPIC_SOURCE_REGION
      This config defines the source region name of the admin topic
      See Also:

    • CHILD_CONTROLLER_ADMIN_TOPIC_CONSUMPTION_ENABLED

      public static final String CHILD_CONTROLLER_ADMIN_TOPIC_CONSUMPTION_ENABLED
      This following config defines whether admin consumption should be enabled or not, and this config will only control the behavior in Child Controller. This is used for store migration.
      See Also:

    • AGGREGATE_REAL_TIME_SOURCE_REGION

      public static final String AGGREGATE_REAL_TIME_SOURCE_REGION
      This config defines the source region of aggregate hybrid store real-time data when native replication is enabled
      See Also:

    • ALLOW_STORE_MIGRATION

      public static final String ALLOW_STORE_MIGRATION
      Whether stores are allowed to be migrated from/to a specific cluster. The value for this config is read from cluster configs in Zk.
      See Also:

    • ALLOW_CLUSTER_WIPE

      public static final String ALLOW_CLUSTER_WIPE
      Whether a cluster in a data center could be wiped. Default is false.
      See Also:

    • CONTROLLER_CLUSTER_HELIX_CLOUD_ENABLED

      public static final String CONTROLLER_CLUSTER_HELIX_CLOUD_ENABLED
      Whether the controller cluster is in a cloud environment. Default is false.
      See Also:

    • CONTROLLER_STORAGE_CLUSTER_HELIX_CLOUD_ENABLED

      public static final String CONTROLLER_STORAGE_CLUSTER_HELIX_CLOUD_ENABLED
      Whether the controller storage cluster is in a cloud environment. Default is false.
      See Also:

    • CONTROLLER_HELIX_CLOUD_PROVIDER

      public static final String CONTROLLER_HELIX_CLOUD_PROVIDER
      What cloud environment the controller is in. Maps to CloudProvider. Default is empty string.
      See Also:

    • CONTROLLER_HELIX_CLOUD_ID

      public static final String CONTROLLER_HELIX_CLOUD_ID
      Specific id in cloud environment that belongs to this cluster. Default is empty string.
      See Also:

    • CONTROLLER_HELIX_CLOUD_INFO_SOURCES

      public static final String CONTROLLER_HELIX_CLOUD_INFO_SOURCES
      Sources for retrieving the cloud information. Default is empty list.
      See Also:

    • CONTROLLER_HELIX_CLOUD_INFO_PROCESSOR_PACKAGE

      public static final String CONTROLLER_HELIX_CLOUD_INFO_PROCESSOR_PACKAGE
      Package name of the class that processes the fetching and parsing of cloud information. Default is empty string.
      See Also:

    • CONTROLLER_HELIX_CLOUD_INFO_PROCESSOR_NAME

      public static final String CONTROLLER_HELIX_CLOUD_INFO_PROCESSOR_NAME
      Name of the class that processes the fetching and parsing of cloud information. Default is empty string.
      See Also:

    • CONTROLLER_HELIX_REST_CUSTOMIZED_HEALTH_URL

      public static final String CONTROLLER_HELIX_REST_CUSTOMIZED_HEALTH_URL
      Base URL for customized health checks triggered by Helix. Default is empty string.
      See Also:

    • CONTROLLER_STORE_GRAVEYARD_CLEANUP_ENABLED

      public static final String CONTROLLER_STORE_GRAVEYARD_CLEANUP_ENABLED
      Whether to enable graveyard cleanup for batch-only store at cluster level. Default is false.
      See Also:

    • CONTROLLER_STORE_GRAVEYARD_CLEANUP_DELAY_MINUTES

      public static final String CONTROLLER_STORE_GRAVEYARD_CLEANUP_DELAY_MINUTES
      When store graveyard cleanup is enabled, delete the graveyard znode if it has not been changed for a specific time. Default is 0 min.
      See Also:

    • CONTROLLER_STORE_GRAVEYARD_CLEANUP_SLEEP_INTERVAL_BETWEEN_LIST_FETCH_MINUTES

      public static final String CONTROLLER_STORE_GRAVEYARD_CLEANUP_SLEEP_INTERVAL_BETWEEN_LIST_FETCH_MINUTES
      Sleep interval between each graveyard list fetch from ZK in StoreGraveyardCleanup service. Default is 15 min.
      See Also:

    • CONTROLLER_PARENT_EXTERNAL_SUPERSET_SCHEMA_GENERATION_ENABLED

      public static final String CONTROLLER_PARENT_EXTERNAL_SUPERSET_SCHEMA_GENERATION_ENABLED
      Whether the superset schema generation in Parent Controller should be done via passed callback or not.
      See Also:

    • CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_SERVICE_ENABLED

      public static final String CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_SERVICE_ENABLED
      Whether to check system store health in parent controller. Default is false.
      See Also:

    • CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_CHECK_INTERVAL_SECONDS

      public static final String CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_CHECK_INTERVAL_SECONDS
      Frequency to run system store health check in parent controller. Default is 30 minute (1800s).
      See Also:

    • CONTROLLER_PARENT_SYSTEM_STORE_HEARTBEAT_CHECK_WAIT_TIME_SECONDS

      public static final String CONTROLLER_PARENT_SYSTEM_STORE_HEARTBEAT_CHECK_WAIT_TIME_SECONDS
      The wait time before validating system store heartbeat during system store health check in parent controller. Default is 1min.
      See Also:

    • CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_RETRY_COUNT

      public static final String CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_RETRY_COUNT
      The maximum retry count for parent controller to fix a bad system store. Default is 1.
      See Also:

    • SYSTEM_SCHEMA_INITIALIZATION_AT_START_TIME_ENABLED

      public static final String SYSTEM_SCHEMA_INITIALIZATION_AT_START_TIME_ENABLED
      Whether to initialize system schemas when controller starts. Default is true.
      See Also:

    • KME_REGISTRATION_FROM_MESSAGE_HEADER_ENABLED

      public static final String KME_REGISTRATION_FROM_MESSAGE_HEADER_ENABLED
      See Also:

    • CONTROLLER_ENABLE_DISABLED_REPLICA_ENABLED

      public static final String CONTROLLER_ENABLE_DISABLED_REPLICA_ENABLED
      The following config is to control whether to turn on disabled replica enabler service.
      See Also:

    • CONTROLLER_DISABLED_REPLICA_ENABLER_INTERVAL_MS

      public static final String CONTROLLER_DISABLED_REPLICA_ENABLER_INTERVAL_MS
      See Also:

    • LISTENER_PORT

      public static final String LISTENER_PORT
      See Also:

    • GRPC_READ_SERVER_PORT

      public static final String GRPC_READ_SERVER_PORT
      See Also:

    • ENABLE_GRPC_READ_SERVER

      public static final String ENABLE_GRPC_READ_SERVER
      See Also:

    • GRPC_SERVER_WORKER_THREAD_COUNT

      public static final String GRPC_SERVER_WORKER_THREAD_COUNT
      See Also:

    • LISTENER_HOSTNAME

      public static final String LISTENER_HOSTNAME
      See Also:

    • DATA_BASE_PATH

      public static final String DATA_BASE_PATH
      See Also:

    • AUTOCREATE_DATA_PATH

      public static final String AUTOCREATE_DATA_PATH
      See Also:

    • ENABLE_SERVER_ALLOW_LIST

      public static final String ENABLE_SERVER_ALLOW_LIST
      See Also:

    • MAX_ONLINE_OFFLINE_STATE_TRANSITION_THREAD_NUMBER

      public static final String MAX_ONLINE_OFFLINE_STATE_TRANSITION_THREAD_NUMBER
      See Also:

    • MAX_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER

      public static final String MAX_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER
      See Also:

    • MAX_FUTURE_VERSION_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER

      public static final String MAX_FUTURE_VERSION_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER
      See Also:

    • LEADER_FOLLOWER_STATE_TRANSITION_THREAD_POOL_STRATEGY

      public static final String LEADER_FOLLOWER_STATE_TRANSITION_THREAD_POOL_STRATEGY
      See Also:

    • STORE_WRITER_NUMBER

      public static final String STORE_WRITER_NUMBER
      See Also:

    • SORTED_INPUT_DRAINER_SIZE

      public static final String SORTED_INPUT_DRAINER_SIZE
      See Also:

    • UNSORTED_INPUT_DRAINER_SIZE

      public static final String UNSORTED_INPUT_DRAINER_SIZE
      See Also:

    • STORE_WRITER_BUFFER_AFTER_LEADER_LOGIC_ENABLED

      public static final String STORE_WRITER_BUFFER_AFTER_LEADER_LOGIC_ENABLED
      See Also:

    • SERVER_INGESTION_TASK_MAX_IDLE_COUNT

      public static final String SERVER_INGESTION_TASK_MAX_IDLE_COUNT
      See Also:

    • STORE_WRITER_BUFFER_MEMORY_CAPACITY

      public static final String STORE_WRITER_BUFFER_MEMORY_CAPACITY
      See Also:

    • STORE_WRITER_BUFFER_NOTIFY_DELTA

      public static final String STORE_WRITER_BUFFER_NOTIFY_DELTA
      See Also:

    • SERVER_REST_SERVICE_STORAGE_THREAD_NUM

      public static final String SERVER_REST_SERVICE_STORAGE_THREAD_NUM
      See Also:

    • SERVER_NETTY_IDLE_TIME_SECONDS

      public static final String SERVER_NETTY_IDLE_TIME_SECONDS
      See Also:

    • SERVER_MAX_REQUEST_SIZE

      public static final String SERVER_MAX_REQUEST_SIZE
      See Also:

    • SERVER_SOURCE_TOPIC_OFFSET_CHECK_INTERVAL_MS

      public static final String SERVER_SOURCE_TOPIC_OFFSET_CHECK_INTERVAL_MS
      See Also:

    • SERVER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS

      public static final String SERVER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS
      See Also:

    • SERVER_NETTY_WORKER_THREADS

      public static final String SERVER_NETTY_WORKER_THREADS
      See Also:

    • SERVER_HELIX_JOIN_AS_UNKNOWN

      public static final String SERVER_HELIX_JOIN_AS_UNKNOWN
      Whether to join a Helix cluster in an UNKNOWN state
      See Also:

    • SSL_TO_KAFKA_LEGACY

      @Deprecated public static final String SSL_TO_KAFKA_LEGACY
      Deprecated.
      Use KAFKA_OVER_SSL
      This config key is a misspelling. It is now considered deprecated.
      See Also:

    • KAFKA_OVER_SSL

      public static final String KAFKA_OVER_SSL
      See Also:

    • SERVER_COMPUTE_THREAD_NUM

      public static final String SERVER_COMPUTE_THREAD_NUM
      See Also:

    • HYBRID_QUOTA_ENFORCEMENT_ENABLED

      public static final String HYBRID_QUOTA_ENFORCEMENT_ENABLED
      See Also:

    • SERVER_DATABASE_MEMORY_STATS_ENABLED

      public static final String SERVER_DATABASE_MEMORY_STATS_ENABLED
      See Also:

    • ROUTER_MAX_READ_CAPACITY

      public static final String ROUTER_MAX_READ_CAPACITY
      See Also:

    • ROUTER_QUOTA_CHECK_WINDOW

      public static final String ROUTER_QUOTA_CHECK_WINDOW
      See Also:

    • ROUTER_USE_LOCAL_SSL_SETTINGS

      public static final String ROUTER_USE_LOCAL_SSL_SETTINGS
      This instructs the router to start running with self signed TLS certificates as opposed to those provided via other properties. This should only be used for testing and defaults to true when running RouterServer.main().
      See Also:

    • ROUTER_ENABLE_SSL

      public static final String ROUTER_ENABLE_SSL
      This instructs the router to open an ssl port. This defaults to true.
      See Also:

    • CONTROLLER_UNUSED_VALUE_SCHEMA_CLEANUP_ENABLED

      public static final String CONTROLLER_UNUSED_VALUE_SCHEMA_CLEANUP_ENABLED
      Enable automatic removal of old unused schemas. Do not enable unless its baked enough, use admin command to delete old schemas.
      See Also:

    • CONTROLLER_UNUSED_SCHEMA_CLEANUP_INTERVAL_SECONDS

      public static final String CONTROLLER_UNUSED_SCHEMA_CLEANUP_INTERVAL_SECONDS
      See Also:

    • CONTROLLER_MIN_SCHEMA_COUNT_TO_KEEP

      public static final String CONTROLLER_MIN_SCHEMA_COUNT_TO_KEEP
      See Also:

    • SERVER_REMOTE_INGESTION_REPAIR_SLEEP_INTERVAL_SECONDS

      public static final String SERVER_REMOTE_INGESTION_REPAIR_SLEEP_INTERVAL_SECONDS
      See Also:

    • SERVER_REST_SERVICE_EPOLL_ENABLED

      public static final String SERVER_REST_SERVICE_EPOLL_ENABLED
      Whether to enable epoll in rest service layer. This will be a best-effort since epoll support is only available in Linux, not Mac.
      See Also:

    • SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_TRANSACTIONAL_MODE

      public static final String SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_TRANSACTIONAL_MODE
      Database sync per bytes for transactional mode. This parameter will impact the sync frequency of database after batch push. For BDB-JE transactional mode, it won't matter since BDB-JE will persist every update in the database right away; For RocksDB transactional mode, it will impact the flush frequency of memtable to SST file, and normally we would like to have this config to be comparable to the memtable size; Negative value will disable this threshold.
      See Also:

    • SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_DEFERRED_WRITE_MODE

      public static final String SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_DEFERRED_WRITE_MODE
      Database sync per bytes for deferred-write mode. This parameter will impact the sync frequency of database during batch push. For BDB-JE deferred-write mode, it will impact the sync frequency, but BDB-JE will do auto-flush if the memory is full; For RocksDB deferred-write mode, it will decide the file size of each SST file since every sync invocation will generate a new SST file; Negative value will disable this threshold.
      See Also:

    • SERVER_PARTITION_GRACEFUL_DROP_DELAY_IN_SECONDS

      public static final String SERVER_PARTITION_GRACEFUL_DROP_DELAY_IN_SECONDS
      When load balance happens, a replica could be moved to another storage node. When dropping the existing replica through Helix state transition: 'ONLINE' -> 'OFFLINE' and 'OFFLINE' -> 'DROPPED', a race condition could happen since Router in-memory partition assignment update through Zookeeper is independent from database drop in storage node, so Router could possibly forward the request to the storage node, which has just dropped the partition. To mitigate this issue, we will add a delay in state transition: 'OFFLINE' -> 'DROPPED' to drain all the incoming requests to the to-drop partition, and we will enable error-retry on Router as well.
      See Also:

    • SERVER_LEAKED_RESOURCE_CLEAN_UP_INTERVAL_IN_MINUTES

      public static final String SERVER_LEAKED_RESOURCE_CLEAN_UP_INTERVAL_IN_MINUTES
      When a BDB partition is dropped, the disk space is not released immediately; a checkpoint is needed to release the disk space; so a cleaner thread is spawned for the entire storage service; the cleaner thread will wake up every few hours and check whether it needs to do a checkpoint; if so, clean up each store sequentially.
      See Also:

    • SERVER_QUOTA_ENFORCEMENT_ENABLED

      public static final String SERVER_QUOTA_ENFORCEMENT_ENABLED
      Set to true to enable enforcement of quota by the storage node
      See Also:

    • SEVER_CALCULATE_QUOTA_USAGE_BASED_ON_PARTITIONS_ASSIGNMENT_ENABLED

      public static final String SEVER_CALCULATE_QUOTA_USAGE_BASED_ON_PARTITIONS_ASSIGNMENT_ENABLED
      Set to true to enable disk quota usage based on partitions assignment reported by the storage node
      See Also:

    • SERVER_NODE_CAPACITY_RCU

      public static final String SERVER_NODE_CAPACITY_RCU
      Number of Read Capacity Units per second that the node can handle across all stores.
      See Also:

    • SERVER_STORE_VERSION_QPS_RATE_LIMITER

      public static final String SERVER_STORE_VERSION_QPS_RATE_LIMITER
      Rate limiter for store version level read quota enforcement.
      See Also:

    • SERVER_STORAGE_NODE_RATE_LIMITER

      public static final String SERVER_STORAGE_NODE_RATE_LIMITER
      Rate limiter for storage node level read quota enforcement.
      See Also:

    • SERVER_QUOTA_ENFORCEMENT_INTERVAL_IN_MILLIS

      public static final String SERVER_QUOTA_ENFORCEMENT_INTERVAL_IN_MILLIS
      Server quota enforcement interval in milliseconds.
      See Also:

    • SERVER_QUOTA_ENFORCEMENT_CAPACITY_MULTIPLE

      public static final String SERVER_QUOTA_ENFORCEMENT_CAPACITY_MULTIPLE
      Server quota enforcement capacity multiple.
      See Also:

    • SERVER_KAFKA_MAX_POLL_RECORDS

      public static final String SERVER_KAFKA_MAX_POLL_RECORDS
      This config is used to control the maximum records returned by every poll request. So far, Store Ingestion is throttling per poll, so if the configured value is too big, the throttling could be inaccurate and it may impact GC as well. We should try to avoid too many long-lasting objects in JVM to minimize GC overhead.
      See Also:

    • SERVER_PUBSUB_CONSUMER_POLL_RETRY_TIMES

      public static final String SERVER_PUBSUB_CONSUMER_POLL_RETRY_TIMES
      This config is used to control how many times PubSub consumer would retry polling during ingestion when RetriableException happens.
      See Also:

    • SERVER_PUBSUB_CONSUMER_POLL_RETRY_BACKOFF_MS

      public static final String SERVER_PUBSUB_CONSUMER_POLL_RETRY_BACKOFF_MS
      This config is used to control the backoff time between PubSub consumer poll retries.
      See Also:

    • SERVER_MAX_WAIT_FOR_VERSION_INFO_MS_CONFIG

      public static final String SERVER_MAX_WAIT_FOR_VERSION_INFO_MS_CONFIG
      Maximum duration (in milliseconds) to wait for the version information to become available in the store metadata repository before skipping Heartbeat (HB) lag monitor setup activity during state transition.
      See Also:

    • SERVER_DISK_HEALTH_CHECK_INTERVAL_IN_SECONDS

      public static final String SERVER_DISK_HEALTH_CHECK_INTERVAL_IN_SECONDS
      This config decides the frequency of the disk health check; the disk health check service writes 64KB data to a temporary file in the database directory and read from the file for each health check.
      See Also:

    • SERVER_DISK_HEALTH_CHECK_TIMEOUT_IN_SECONDS

      public static final String SERVER_DISK_HEALTH_CHECK_TIMEOUT_IN_SECONDS
      When there is an actual disk failure, health check operation would hang, so this config decides how fast the servers will start reporting unhealthy after the health check stop updating status; however, in order to reduce the possibility of false alerts (for example, the health check updates can be delayed by GC), we couldn't set the timeout too small. Currently by default, the timeout is 30 seconds.
      See Also:

    • SERVER_DISK_HEALTH_CHECK_SERVICE_ENABLED

      public static final String SERVER_DISK_HEALTH_CHECK_SERVICE_ENABLED
      This config is used to enable/disable the disk health check service.
      See Also:

    • SERVER_COMPUTE_FAST_AVRO_ENABLED

      public static final String SERVER_COMPUTE_FAST_AVRO_ENABLED
      Whether to enable fast-avro in compute request path.
      See Also:

    • SERVER_ENABLE_PARALLEL_BATCH_GET

      public static final String SERVER_ENABLE_PARALLEL_BATCH_GET
      Whether to enable parallel lookup for batch-get.
      See Also:

    • SERVER_PARALLEL_BATCH_GET_CHUNK_SIZE

      public static final String SERVER_PARALLEL_BATCH_GET_CHUNK_SIZE
      Chunk size of each task for parallel lookup of batch-get.
      See Also:

    • SERVER_STORE_TO_EARLY_TERMINATION_THRESHOLD_MS_MAP

      public static final String SERVER_STORE_TO_EARLY_TERMINATION_THRESHOLD_MS_MAP
      The request early termination threshold map: The key will be store name, and the value will be the actual threshold. This config is temporary, and in the long run, we will ask Venice Client to pass the actual timeout to the backend.
      See Also:

    • SERVER_DATABASE_LOOKUP_QUEUE_CAPACITY

      public static final String SERVER_DATABASE_LOOKUP_QUEUE_CAPACITY
      The following config is used to control the maximum database lookup requests queued, when the queue is full, server will propagate the back pressure to the caller.
      See Also:

    • SERVER_COMPUTE_QUEUE_CAPACITY

      public static final String SERVER_COMPUTE_QUEUE_CAPACITY
      Check @SERVER_DATABASE_LOOKUP_QUEUE_CAPACITY for the explanation. The following config is specifically for compute request.
      See Also:

    • SERVER_BLOCKING_QUEUE_TYPE

      public static final String SERVER_BLOCKING_QUEUE_TYPE
      Check the available types in com.linkedin.venice.config.BlockingQueueType
      See Also:

    • SERVER_ROUTER_CONNECTION_WARMING_DELAY_MS

      public static final String SERVER_ROUTER_CONNECTION_WARMING_DELAY_MS
      This config is used to control how much time Server will wait for connection warming from Routers. This is trying to avoid availability issue when router connection warming happens when Server restarts. In theory, this config should be equal to or bigger than ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_NEW_INSTANCE_DELAY_JOIN_MS.
      See Also:

    • SERVER_CONSUMER_POOL_SIZE_PER_KAFKA_CLUSTER

      public static final String SERVER_CONSUMER_POOL_SIZE_PER_KAFKA_CLUSTER
      Consumer pool size per Kafka cluster.
      See Also:

    • MIN_CONSUMER_IN_CONSUMER_POOL_PER_KAFKA_CLUSTER

      public static final String MIN_CONSUMER_IN_CONSUMER_POOL_PER_KAFKA_CLUSTER
      See Also:

    • SERVER_SHARED_CONSUMER_ASSIGNMENT_STRATEGY

      public static final String SERVER_SHARED_CONSUMER_ASSIGNMENT_STRATEGY
      Whether to enable partition wise balanced shared consumer assignment.
      See Also:

    • SERVER_LEAKED_RESOURCE_CLEANUP_ENABLED

      public static final String SERVER_LEAKED_RESOURCE_CLEANUP_ENABLED
      Whether to enable leaked resource cleanup in storage node. Right now, it only covers leaked storage partitions on disk.
      See Also:

    • SERVER_INGESTION_MODE

      public static final String SERVER_INGESTION_MODE
      Ingestion mode in target storage instance. This will be applied to Da Vinci and Storage Node.
      See Also:

    • SERVER_UNSUB_AFTER_BATCHPUSH

      public static final String SERVER_UNSUB_AFTER_BATCHPUSH
      Unsubscribe from kakfa topic once a batch-store push finishes
      See Also:

    • SERVER_DEDICATED_DRAINER_FOR_SORTED_INPUT_ENABLED

      public static final String SERVER_DEDICATED_DRAINER_FOR_SORTED_INPUT_ENABLED
      Use a seprate drainer queue for sorted ingestion and un-sorted ingestion.
      See Also:

    • INGESTION_USE_DA_VINCI_CLIENT

      public static final String INGESTION_USE_DA_VINCI_CLIENT
      A boolean config to specify if we are using Da Vinci client for ingestion. This config will be parsed by isDaVinciConfig variable in VeniceServerConfig. By default, it is false (use Venice Server)
      See Also:

    • SERVER_STOP_CONSUMPTION_TIMEOUT_IN_SECONDS

      public static final String SERVER_STOP_CONSUMPTION_TIMEOUT_IN_SECONDS
      Number of retries allowed for stopConsumptionAndWait() API in StoreIngestionService.
      See Also:

    • SERVER_INGESTION_ISOLATION_SERVICE_PORT

      public static final String SERVER_INGESTION_ISOLATION_SERVICE_PORT
      Service listening port number for main ingestion service.
      See Also:

    • SERVER_INGESTION_ISOLATION_APPLICATION_PORT

      public static final String SERVER_INGESTION_ISOLATION_APPLICATION_PORT
      Service listening port number for forked ingestion process.
      See Also:

    • SERVER_DB_READ_ONLY_FOR_BATCH_ONLY_STORE_ENABLED

      public static final String SERVER_DB_READ_ONLY_FOR_BATCH_ONLY_STORE_ENABLED
      See Also:

    • SERVER_RESET_ERROR_REPLICA_ENABLED

      public static final String SERVER_RESET_ERROR_REPLICA_ENABLED
      See Also:

    • SERVER_ADAPTIVE_THROTTLER_ENABLED

      public static final String SERVER_ADAPTIVE_THROTTLER_ENABLED
      See Also:

    • SERVER_ADAPTIVE_THROTTLER_SIGNAL_IDLE_THRESHOLD

      public static final String SERVER_ADAPTIVE_THROTTLER_SIGNAL_IDLE_THRESHOLD
      See Also:

    • SERVER_ADAPTIVE_THROTTLER_SINGLE_GET_LATENCY_THRESHOLD

      public static final String SERVER_ADAPTIVE_THROTTLER_SINGLE_GET_LATENCY_THRESHOLD
      See Also:

    • SERVER_INGESTION_ISOLATION_STATS_CLASS_LIST

      public static final String SERVER_INGESTION_ISOLATION_STATS_CLASS_LIST
      A list of fully-qualified class names of all stats classes that needs to be initialized in isolated ingestion process, separated by comma. This config will help isolated ingestion process to register extra stats needed for monitoring, for example: JVM GC/Memory stats. All the classes defined here will be extending AbstractVeniceStats, and will take MetricsRepository as the only parameter in their constructor.
      See Also:

    • SERVER_INGESTION_ISOLATION_SSL_ENABLED

      public static final String SERVER_INGESTION_ISOLATION_SSL_ENABLED
      See Also:

    • SERVER_INGESTION_ISOLATION_ACL_ENABLED

      public static final String SERVER_INGESTION_ISOLATION_ACL_ENABLED
      See Also:

    • SERVER_INGESTION_ISOLATION_PRINCIPAL_NAME

      public static final String SERVER_INGESTION_ISOLATION_PRINCIPAL_NAME
      See Also:

    • SERVER_FORKED_PROCESS_JVM_ARGUMENT_LIST

      public static final String SERVER_FORKED_PROCESS_JVM_ARGUMENT_LIST
      A list of JVM arguments for forked child process, separated by semicolon.
      See Also:

    • SERVER_INGESTION_ISOLATION_CONNECTION_TIMEOUT_SECONDS

      public static final String SERVER_INGESTION_ISOLATION_CONNECTION_TIMEOUT_SECONDS
      Timeout for connection between main process and forked ingestion process. If heartbeat is not refreshed within this timeout, both processes should act to reconstruct the state in order to restore connection and service.
      See Also:

    • SERVER_INGESTION_ISOLATION_REQUEST_TIMEOUT_SECONDS

      public static final String SERVER_INGESTION_ISOLATION_REQUEST_TIMEOUT_SECONDS
      Timeout for single ingestion command request sent from main process to forked ingestion process.
      See Also:

    • SERVER_INGESTION_ISOLATION_HEARTBEAT_REQUEST_TIMEOUT_SECONDS

      public static final String SERVER_INGESTION_ISOLATION_HEARTBEAT_REQUEST_TIMEOUT_SECONDS
      Timeout for single heartbeat request sent from main process to forked ingestion process.
      See Also:

    • SERVER_BATCH_REPORT_END_OF_INCREMENTAL_PUSH_STATUS_ENABLED

      public static final String SERVER_BATCH_REPORT_END_OF_INCREMENTAL_PUSH_STATUS_ENABLED
      See Also:

    • SERVER_INCREMENTAL_PUSH_STATUS_WRITE_MODE

      public static final String SERVER_INCREMENTAL_PUSH_STATUS_WRITE_MODE
      This config dictates where the server should write the end of incremental push status.
      See Also:

    • SERVER_DATABASE_CHECKSUM_VERIFICATION_ENABLED

      public static final String SERVER_DATABASE_CHECKSUM_VERIFICATION_ENABLED
      whether to enable checksum verification in the ingestion path from kafka to database persistency. If enabled it will keep a running checksum for all and only PUT kafka data message received in the ingestion task and periodically verify it against the key/values saved in the database persistency layer.
      See Also:

    • SERVER_LOCAL_CONSUMER_CONFIG_PREFIX

      public static final String SERVER_LOCAL_CONSUMER_CONFIG_PREFIX
      Any server config that start with "server.local.consumer.config.prefix" will be used as a customized consumer config for local consumer.
      See Also:

    • SERVER_REMOTE_CONSUMER_CONFIG_PREFIX

      public static final String SERVER_REMOTE_CONSUMER_CONFIG_PREFIX
      Any server config that start with "server.remote.consumer.config.prefix" will be used as a customized consumer config for remote consumer.
      See Also:

    • SERVER_ROCKSDB_STORAGE_CONFIG_CHECK_ENABLED

      public static final String SERVER_ROCKSDB_STORAGE_CONFIG_CHECK_ENABLED
      Whether to enable to check the RocksDB storage type used to open the RocksDB storage engine and how it was built. Having different storage types (BlockBasedTable and PlainTable) in read ops and write ops may lead to corruption of RocksDB storage and crash of servers.
      See Also:

    • SERVER_SHARED_CONSUMER_NON_EXISTING_TOPIC_CLEANUP_DELAY_MS

      public static final String SERVER_SHARED_CONSUMER_NON_EXISTING_TOPIC_CLEANUP_DELAY_MS
      This config is used to control how much time we should wait before cleaning up the corresponding ingestion task when an non-existing topic is discovered. The reason to introduce this config is that `consumer#listTopics` could only guarantee eventual consistency, so `consumer#listTopics` not returning the topic doesn't mean the topic doesn't exist in Kafka. If `consumer#listTopics` still doesn't return the topic after the configured delay, Venice SN will unsubscribe the topic, and fail the corresponding ingestion job.
      See Also:

    • FREEZE_INGESTION_IF_READY_TO_SERVE_OR_LOCAL_DATA_EXISTS

      public static final String FREEZE_INGESTION_IF_READY_TO_SERVE_OR_LOCAL_DATA_EXISTS
      This config will determine whether live update will be suppressed. When the feature is turned on, ingestion will stop once a partition is ready to serve; after Da Vinci client restarts or server restarts, if local data exists, ingestion will not start in Da Vinci or report ready-to-serve immediately without ingesting new data in Venice.
      See Also:

    • KAFKA_PRODUCER_METRICS

      public static final String KAFKA_PRODUCER_METRICS
      a comma seperated list of kafka producer metrics that will be reported. For ex. "outgoing-byte-rate,record-send-rate,batch-size-max,batch-size-avg,buffer-available-bytes,buffer-exhausted-rate"
      See Also:

    • SERVER_DEBUG_LOGGING_ENABLED

      public static final String SERVER_DEBUG_LOGGING_ENABLED
      Whether to print logs that are used for troubleshooting only.
      See Also:

    • SERVER_NUM_SCHEMA_FAST_CLASS_WARMUP

      public static final String SERVER_NUM_SCHEMA_FAST_CLASS_WARMUP
      Number of value schemas for which fast avro classes are generated for read-compute stores before push completion is reported.
      See Also:

    • SERVER_SCHEMA_FAST_CLASS_WARMUP_TIMEOUT

      public static final String SERVER_SCHEMA_FAST_CLASS_WARMUP_TIMEOUT
      Timeout duration of schema generation for fast class warmup period
      See Also:

    • LOCAL_CONTROLLER_URL

      public static final String LOCAL_CONTROLLER_URL
      The following 3 configs define controller url, d2 service name and d2 zk host in the region that server is located. Either url or d2 configs must be specified if SYSTEM_SCHEMA_INITIALIZATION_AT_START_TIME_ENABLED is true.
      See Also:

    • LOCAL_CONTROLLER_D2_SERVICE_NAME

      public static final String LOCAL_CONTROLLER_D2_SERVICE_NAME
      See Also:

    • LOCAL_D2_ZK_HOST

      public static final String LOCAL_D2_ZK_HOST
      See Also:

    • LISTENER_SSL_PORT

      public static final String LISTENER_SSL_PORT
      See Also:

    • HEARTBEAT_TIMEOUT

      public static final String HEARTBEAT_TIMEOUT
      See Also:

    • HEARTBEAT_CYCLE

      public static final String HEARTBEAT_CYCLE
      See Also:

    • MAX_READ_CAPACITY

      public static final String MAX_READ_CAPACITY
      See Also:

    • SSL_TO_STORAGE_NODES

      public static final String SSL_TO_STORAGE_NODES
      See Also:

    • ROUTER_LONG_TAIL_RETRY_FOR_SINGLE_GET_THRESHOLD_MS

      public static final String ROUTER_LONG_TAIL_RETRY_FOR_SINGLE_GET_THRESHOLD_MS
      After this amount of time, DDS Router will retry once for the slow storage node request. Practically, we need to manually select the threshold (e.g. P95) for retrying based on latency metrics.
      See Also:

    • ROUTER_LONG_TAIL_RETRY_FOR_BATCH_GET_THRESHOLD_MS

      public static final String ROUTER_LONG_TAIL_RETRY_FOR_BATCH_GET_THRESHOLD_MS
      After this amount of time, DDS Router will retry once for the slow storage node request. The configured format will be like this way: "1-10:20,11-50:50,51-200:80,201-:1000" Let me explain the config by taking one example: If the request key count for a batch-get request is '32', and it fails into this key range: [11-50], so the retry threshold for this batch-get request is 50ms. That is a limitation here: The retry threshold is actually for each scatter-gather request, but this config is not strictly with the actual key count inside each scatter-gather request, which means even if there is only one key in a scatter-gather request with the above example, Router will wait for 50ms to retry this scatter-gather request. For now, it is not big issue since for now we mostly want to use this config to skip the storage node, which is experiencing long GC pause. So coarse-grained config should be good enough.
      See Also:

    • ROUTER_SMART_LONG_TAIL_RETRY_ENABLED

      public static final String ROUTER_SMART_LONG_TAIL_RETRY_ENABLED
      Whether to enable smart long tail retry logic, and this logic is only useful for batch-get retry currently. This feature is used to avoid the unnecessary retries in the following scenarios: 1. Router is suffering long GC pause, no matter whether Storage Node is fast or not; 2. The retried Storage Node is slow according to the original request; For case 1, unnecessary retries will make Router GC behavior even worse; For case 2, unnecessary retries to the slow Storage Node will make the slow Storage Node even slower, and the overall latency won't be improved; For case 1, here is how smart retry works: 1. When the delay between the retry request and the original request is over ROUTER_LONG_TAIL_RETRY_FOR_BATCH_GET_THRESHOLD_MS + ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS, smart retry logic will treat the current Router to be in bad state (long GC pause or too busy), the retry request will be aborted; 2.ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS is the way to measure whether Router is in good state or not, and need to be tuned in prod; For case 2, the retry request will be aborted if the original request to the same storage node hasn't returned, and the slowness measurement is inside one request when scatter-gathering.
      See Also:

    • ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS

      public static final String ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS
      This config is used to tune the smart long-tail retry logic to avoid unnecessary retries, check more details: ROUTER_SMART_LONG_TAIL_RETRY_ENABLED
      See Also:

    • ROUTER_LONG_TAIL_RETRY_MAX_ROUTE_FOR_MULTI_KEYS_REQ

      public static final String ROUTER_LONG_TAIL_RETRY_MAX_ROUTE_FOR_MULTI_KEYS_REQ
      This config is used to limit the maximum retries in route unit. In large batch-get/compute cluster, when enabling long-tail retry, in the worst scenarios, Router could trigger a retry storm since each route could retry independently. This config is used to specify the maximum retry in route unit. If the configured value is 1, it means the current request will at most one route. This could mitigate the latency issue in most of the case since the chance to have multiple slow storage nodes is low, also even with unlimited retries, it won't help since multiple replicas for the same partition are in a degraded state.
      See Also:

    • ROUTER_MAX_KEY_COUNT_IN_MULTIGET_REQ

      public static final String ROUTER_MAX_KEY_COUNT_IN_MULTIGET_REQ
      The max key count allowed in one multi-get request. For now, it is configured in host level, and we could consider to configure it in store level.
      See Also:

    • ROUTER_CONNECTION_LIMIT

      public static final String ROUTER_CONNECTION_LIMIT
      See Also:

    • ROUTER_HTTP_CLIENT_POOL_SIZE

      public static final String ROUTER_HTTP_CLIENT_POOL_SIZE
      The http client pool size being used in one Router;
      See Also:

    • ROUTER_IO_WORKER_COUNT

      public static final String ROUTER_IO_WORKER_COUNT
      Config to control the Netty IO thread count for the Router Server
      See Also:

    • ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE

      public static final String ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE
      The max connection number per route (to one storage node);
      See Also:

    • ROUTER_MAX_OUTGOING_CONNECTION

      public static final String ROUTER_MAX_OUTGOING_CONNECTION
      The max connection number in one Router to storage nodes;
      See Also:

    • ROUTER_MAX_PENDING_REQUEST

      public static final String ROUTER_MAX_PENDING_REQUEST
      This config is used to bound the pending request. Without this config, the accumulated requests in Http Async Client could grow unlimitedly, which would put Router in a non-recoverable state because of long GC pause introduced by the increasing memory usage. If the incoming request exceeds this configured threshold, Router will return 503 (Service Unavailable).
      See Also:

    • ROUTER_MULTI_KEY_ROUTING_STRATEGY

      public static final String ROUTER_MULTI_KEY_ROUTING_STRATEGY
      This config is used to define the routing strategy for multi-key requests. Please check VeniceMultiKeyRoutingStrategy to find available routing strategy.
      See Also:

    • ROUTER_HELIX_VIRTUAL_GROUP_FIELD_IN_DOMAIN

      public static final String ROUTER_HELIX_VIRTUAL_GROUP_FIELD_IN_DOMAIN
      The Helix virtual group field name in domain, and the allowed values: HelixInstanceConfigRepository.GROUP_FIELD_NAME_IN_DOMAIN and HelixInstanceConfigRepository.ZONE_FIELD_NAME_IN_DOMAIN.
      See Also:

    • ROUTER_HELIX_ASSISTED_ROUTING_GROUP_SELECTION_STRATEGY

      public static final String ROUTER_HELIX_ASSISTED_ROUTING_GROUP_SELECTION_STRATEGY
      Helix group selection strategy when Helix assisted routing is enabled. Available strategies listed here: HelixGroupSelectionStrategyEnum.
      See Also:

    • ROUTER_PER_STORE_ROUTER_QUOTA_BUFFER

      public static final String ROUTER_PER_STORE_ROUTER_QUOTA_BUFFER
      See Also:

    • ROUTER_DNS_CACHE_ENABLED

      public static final String ROUTER_DNS_CACHE_ENABLED
      Whether to enable customized dns cache in router or not. This is mostly to address slow DNS lookup issue.
      See Also:

    • ROUTE_DNS_CACHE_HOST_PATTERN

      public static final String ROUTE_DNS_CACHE_HOST_PATTERN
      The host matching the configured host pattern will be cached if ROUTER_DNS_CACHE_ENABLED is true.
      See Also:

    • ROUTER_DNS_CACHE_REFRESH_INTERVAL_MS

      public static final String ROUTER_DNS_CACHE_REFRESH_INTERVAL_MS
      Refresh interval of cached dns entries if ROUTER_DNS_CACHE_ENABLED is true.
      See Also:

    • ROUTER_STORAGE_NODE_CLIENT_TYPE

      public static final String ROUTER_STORAGE_NODE_CLIENT_TYPE
      Whether the router use netty http client or apache http async client
      See Also:

    • ROUTER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS

      public static final String ROUTER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS
      Netty graceful shutdown period considering the following factors: 1. D2 de-announcement could take some time; 2. Client could take some time to receive/apply the zk update event from D2 server about router shutdown; 3. Router needs some time to handle already-received client requests;
      See Also:

    • ROUTER_CLIENT_DECOMPRESSION_ENABLED

      public static final String ROUTER_CLIENT_DECOMPRESSION_ENABLED
      See Also:

    • ROUTER_SOCKET_TIMEOUT

      public static final String ROUTER_SOCKET_TIMEOUT
      Socket timeout config for the connection manager from router to server
      See Also:

    • ROUTER_CONNECTION_TIMEOUT

      public static final String ROUTER_CONNECTION_TIMEOUT
      Timeout for building a new connection from router to server
      See Also:

    • ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_ENABLED

      public static final String ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_ENABLED
      Whether to enable the cleanup of the idle connections to storage node. Recently, we are seeing latency spike because of new connection setup, and we hope the total available connections will be more stable by disabling the idle connection cleanup. The potential long-term solutions could be connection warm-up for HTTP/1.1 or adopting HTTP/2
      See Also:

    • ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_THRESHOLD_MINS

      public static final String ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_THRESHOLD_MINS
      The idle threshold for cleaning up the connections to storage node.
      See Also:

    • ROUTER_FULL_PENDING_QUEUE_SERVER_OOR_MS

      public static final String ROUTER_FULL_PENDING_QUEUE_SERVER_OOR_MS
      The following config controls how long the server with full pending queue will be taken OOR.
      See Also:

    • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_ENABLED

      public static final String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_ENABLED
      Connection warming feature for httpasynclient. So far, it only works when Router starts and runs in http-client-per-route mode, and it will try to warm up ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE connections per route.
      See Also:

    • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_SLEEP_INTERVAL_MS

      public static final String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_SLEEP_INTERVAL_MS
      When Router starts, for a given route, the following config controls the warming up speed to minimize the impact to storage nodes.
      See Also:

    • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_LOW_WATER_MARK

      public static final String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_LOW_WATER_MARK
      When the available connections in an httpasyncclient is below the low water mark, the connection warming service will try to spin up a new client to replace it. In theory, this config must be lower than ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE.
      See Also:

    • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_EXECUTOR_THREAD_NUM

      public static final String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_EXECUTOR_THREAD_NUM
      Connection warming executor thread num.
      See Also:

    • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_NEW_INSTANCE_DELAY_JOIN_MS

      public static final String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_NEW_INSTANCE_DELAY_JOIN_MS
      For the new instance (Storage Node) detected by Router, the following config defines how much delay because of connection warming it could tolerate. If the connection warming takes longer than it, Router will put it in to serve online traffic by creating a new client without connection warming.
      See Also:

    • ROUTER_HTTPAYSNCCLIENT_CONNECTION_WARMING_SOCKET_TIMEOUT_MS

      public static final String ROUTER_HTTPAYSNCCLIENT_CONNECTION_WARMING_SOCKET_TIMEOUT_MS
      This config is used to control the socket timeout for connection warming requests. In some cases, we would like to have different(maybe longer timeout) than the regular requests considering the deployment procedure, and the connection warming requests could be very instensive.
      See Also:

    • ROUTER_ASYNC_START_ENABLED

      public static final String ROUTER_ASYNC_START_ENABLED
      Whether to enable async start in Router startup procedure. The reason to introduce this feature is that in some env, the dependent services could be started out of order. IMPORTANT: enabling this feature won't guarantee that a successful restarted Router will be in a healthy state, since async start will exit later if it detects any errors.
      See Also:

    • CONTROLLER_CLUSTER_REPLICA

      public static final String CONTROLLER_CLUSTER_REPLICA
      Venice uses a helix cluster to assign controllers to each named venice cluster. This is the number of controllers assigned to each venice cluster. Should normally be 3; one leader controller and 2 standby controllers.
      See Also:

    • MIN_NUMBER_OF_UNUSED_KAFKA_TOPICS_TO_PRESERVE

      public static final String MIN_NUMBER_OF_UNUSED_KAFKA_TOPICS_TO_PRESERVE
      This is the minimum number of Kafka topics that are guaranteed to be preserved by the leaky topic clean up routine. The topics with the highest version numbers will be favored by this preservative behavior. All other topics (i.e.: those with smaller version numbers) which Venice does not otherwise know about from its metadata will be considered leaked resources and thus be eligible for clean up. A value greater than zero is recommended for Storage Node to not run into UNKNOWN_TOPIC_OR_PARTITION issues N.B.: A known limitation of this preservation setting is that during store deletion, if a topic has been leaked recently due to an aborted push, then there is an edge case where that topic may leak forever. This leak does not happen if the latest store-versions are successful pushes, rather than failed ones. Furthermore, if a store with the same name is ever re-created, then the clean up routine would resume and clean up the older leaky topics successfully. This edge case is deemed a small enough concern for now, though it could be addressed with a more significant redesign of the replication pipeline.
      See Also:

    • MIN_NUMBER_OF_STORE_VERSIONS_TO_PRESERVE

      public static final String MIN_NUMBER_OF_STORE_VERSIONS_TO_PRESERVE
      This is the number of fully-functional store-versions we wish to maintain. All resources of these versions will be preserved (Helix resource, Storage Node data, Kafka topic, replication streams, etc.), and a swap to these versions should be possible at all times. This setting must be set to 1 or greater.
      See Also:

    • CONTROLLER_PARENT_MODE

      public static final String CONTROLLER_PARENT_MODE
      Whether current controller is parent or not
      See Also:

    • CONTROLLER_PARENT_REGION_STATE

      public static final String CONTROLLER_PARENT_REGION_STATE
      This config specifies the state of the region of the parent controller. The region can be in one of the following states: ACTIVE: the parent controller in the region is serving requests. PASSIVE: the parent controller in the region is rejecting requests. By default, this is set to ACTIVE.
      See Also:

    • PARENT_CONTROLLER_MAX_ERRORED_TOPIC_NUM_TO_KEEP

      public static final String PARENT_CONTROLLER_MAX_ERRORED_TOPIC_NUM_TO_KEEP
      This config is used to control how many errored topics we are going to keep in parent cluster. This is mostly used to investigate the Kafka missing message issue. If the issue gets resolved, we could change this config to be '0'.
      See Also:

    • CHILD_CLUSTER_URL_PREFIX

      public static final String CHILD_CLUSTER_URL_PREFIX
      Only required when controller.parent.mode=true This prefix specifies the location of every child cluster that is being fed by this parent cluster. The format for key/value would be like "key=child.cluster.url.dc-0, value=url1;url2;url3" the cluster name should be human-readable, ex: dc-0 the url should be of the form http://host:port Note that every cluster name supplied must also be specified in the child.cluster.allowlist in order to be included
      See Also:

    • CHILD_CLUSTER_D2_PREFIX

      public static final String CHILD_CLUSTER_D2_PREFIX
      Similar to CHILD_CLUSTER_URL_PREFIX but with D2 ZK url.
      See Also:

    • CHILD_DATA_CENTER_KAFKA_URL_PREFIX

      public static final String CHILD_DATA_CENTER_KAFKA_URL_PREFIX
      Config prefix for Kafka bootstrap url in all child fabrics; parent controllers need to know the Kafka url in all fabrics for native replication.
      See Also:

    • CHILD_CLUSTER_D2_SERVICE_NAME

      public static final String CHILD_CLUSTER_D2_SERVICE_NAME
      D2 Service name for the child controllers in local datacenter
      See Also:

    • CLUSTER_DISCOVERY_D2_SERVICE

      public static final String CLUSTER_DISCOVERY_D2_SERVICE
      D2 Service name for cluster discovery
      See Also:

    • NATIVE_REPLICATION_SOURCE_FABRIC

      public static final String NATIVE_REPLICATION_SOURCE_FABRIC
      The default source fabric used for native replication
      See Also:

    • NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_BATCH_ONLY_STORES

      public static final String NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_BATCH_ONLY_STORES
      The default source fabric used for native replication for batch only stores.
      See Also:

    • NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_HYBRID_STORES

      public static final String NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_HYBRID_STORES
      The default source fabric used for native replication for hybrid stores.
      See Also:

    • ENABLE_INCREMENTAL_PUSH_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES

      public static final String ENABLE_INCREMENTAL_PUSH_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES
      We will use this config to determine whether we should enable incremental push for hybrid active-active user stores. If this config is set to true, we will enable incremental push for hybrid active-active user stores.
      See Also:

    • ENABLE_SEPARATE_REAL_TIME_TOPIC_FOR_STORE_WITH_INCREMENTAL_PUSH

      public static final String ENABLE_SEPARATE_REAL_TIME_TOPIC_FOR_STORE_WITH_INCREMENTAL_PUSH
      We will use this config to determine whether we should enable separate real-time topic for incremental push enabled stores. If this config is set to true, we will enable separate real-time topic for incremental push enabled stores.
      See Also:

    • ENABLE_PARTIAL_UPDATE_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES

      public static final String ENABLE_PARTIAL_UPDATE_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES
      We will use this config to determine whether we should enable partial update for hybrid active-active user stores. If this config is set to true, we will enable partial update for hybrid active-active user stores whose latest value schema meets partial update feature requirement.
      See Also:

    • ENABLE_PARTIAL_UPDATE_FOR_HYBRID_NON_ACTIVE_ACTIVE_USER_STORES

      public static final String ENABLE_PARTIAL_UPDATE_FOR_HYBRID_NON_ACTIVE_ACTIVE_USER_STORES
      We will use this config to determine whether we should enable partial update for hybrid non-active-active user stores. If this config is set to true, we will enable partial update for hybrid active-active user stores whose latest value schema meets partial update feature requirement.
      See Also:

    • EMERGENCY_SOURCE_REGION

      public static final String EMERGENCY_SOURCE_REGION
      The highest priority source fabric selection config, specified in parent controller.
      See Also:

    • CHILD_CLUSTER_WHITELIST

      @Deprecated public static final String CHILD_CLUSTER_WHITELIST
      Deprecated.
      See Also:

    • CHILD_CLUSTER_ALLOWLIST

      public static final String CHILD_CLUSTER_ALLOWLIST
      Only required when controller.parent.mode=true This is a comma-separated allowlist of cluster names used in the keys with the child.cluster.url prefix. Example, if we have the following child.cluster.url keys: child.cluster.url.cluster1=... child.cluster.url.cluster2=... child.cluster.url.cluster3=... And we want to use all three cluster, then we set child.cluster.allowlist=cluster1,cluster2,cluster3 If we only want to use clusters 1 and 3 we can set child.cluster.allowlist=cluster1,cluster3
      See Also:

    • NATIVE_REPLICATION_FABRIC_WHITELIST

      @Deprecated public static final String NATIVE_REPLICATION_FABRIC_WHITELIST
      Deprecated.
      See Also:

    • NATIVE_REPLICATION_FABRIC_ALLOWLIST

      public static final String NATIVE_REPLICATION_FABRIC_ALLOWLIST
      Previously CHILD_CLUSTER_ALLOWLIST is used to also represent the allowlist of source fabrics for native replication; however, the final migration plan decides that a Kafka cluster in parent fabric can also be the source fabric, so the below config is introduced to represent all potential source fabrics for native replication.
      See Also:

    • PARENT_KAFKA_CLUSTER_FABRIC_LIST

      public static final String PARENT_KAFKA_CLUSTER_FABRIC_LIST
      A list of potential parent fabrics. Logically, there is only one parent fabric; during native replication migration, there will be two Kafka clusters in parent fabric, so we need two fabric names to represent the two different Kafka cluster url. TODO: deprecate this config after native replication migration is complete.
      See Also:

    • ACTIVE_ACTIVE_REAL_TIME_SOURCE_FABRIC_LIST

      public static final String ACTIVE_ACTIVE_REAL_TIME_SOURCE_FABRIC_LIST
      A list of regions that are source(s) of the Active/Active real time replication. When running in a multi-region mode, this list should contain region names where the Venice server should consume from when it accepts the TS (TopicSwitch) message. Example value of this config: "dc-0, dc-1, dc-2".
      See Also:

    • PARENT_CONTROLLER_WAITING_TIME_FOR_CONSUMPTION_MS

      public static final String PARENT_CONTROLLER_WAITING_TIME_FOR_CONSUMPTION_MS
      When the parent controller receives an admin write operation, it replicates that message to the admin kafka stream. After replication the parent controller consumes the message from the stream and processes it there. This is the timeout for waiting until that consumption happens.
      See Also:

    • ADMIN_CONSUMPTION_CYCLE_TIMEOUT_MS

      public static final String ADMIN_CONSUMPTION_CYCLE_TIMEOUT_MS
      The maximum time allowed for worker threads to execute admin messages in one cycle. A cycle is the processing of delegated admin messages by some number of worker thread(s) defined by ADMIN_CONSUMPTION_MAX_WORKER_THREAD_POOL_SIZE. Each worker thread will try to empty the queue for a store before moving on to process admin messages for another store. The cycle is completed either by finishing all delegated admin messages or timing out with this config. TODO: Note that the timeout is for all stores in the cycle and not individual stores. Meaning that some stores may starve.
      See Also:

    • ADMIN_CONSUMPTION_MAX_WORKER_THREAD_POOL_SIZE

      public static final String ADMIN_CONSUMPTION_MAX_WORKER_THREAD_POOL_SIZE
      The maximum number of threads allowed in the pool for executing admin messages.
      See Also:

    • STORAGE_ENGINE_OVERHEAD_RATIO

      public static final String STORAGE_ENGINE_OVERHEAD_RATIO
      This factor is used to estimate potential push size. VPJ reducer multiplies it with total record size and compares it with store storage quota TODO: it will be moved to Store metadata if we allow stores have various storage engine types.
      See Also:

    • ENABLE_OFFLINE_PUSH_SSL_WHITELIST

      @Deprecated public static final String ENABLE_OFFLINE_PUSH_SSL_WHITELIST
      Deprecated.
      See Also:

    • ENABLE_OFFLINE_PUSH_SSL_ALLOWLIST

      public static final String ENABLE_OFFLINE_PUSH_SSL_ALLOWLIST
      The switcher to enable/disable the allowlist of ssl offline pushes. If we disable the allowlist here, depends on the config "SSL_TO_KAFKA", all pushes will be secured by SSL or none of pushes will be secured by SSL.
      See Also:

    • ENABLE_HYBRID_PUSH_SSL_WHITELIST

      @Deprecated public static final String ENABLE_HYBRID_PUSH_SSL_WHITELIST
      Deprecated.
      See Also:

    • ENABLE_HYBRID_PUSH_SSL_ALLOWLIST

      public static final String ENABLE_HYBRID_PUSH_SSL_ALLOWLIST
      The switcher to enable/disable the allowlist of ssl hybrid pushes including both batch and near-line pushes for that store. If we disable the allowlist here, depends on the config "SSL_TO_KAFKA", all pushes will be secured by SSL or none of pushes will be secured by SSL.
      See Also:

    • PUSH_SSL_WHITELIST

      @Deprecated public static final String PUSH_SSL_WHITELIST
      Deprecated.
      See Also:

    • PUSH_SSL_ALLOWLIST

      public static final String PUSH_SSL_ALLOWLIST
      Allowlist of stores which are allowed to push data with SSL.
      See Also:

    • ENFORCE_SECURE_ROUTER

      public static final String ENFORCE_SECURE_ROUTER
      Whether to block storage requests on the non-ssl port. Will still allow metadata requests on the non-ssl port and will log storage requests on the non-ssl port even if set to false;
      See Also:

    • HELIX_REBALANCE_ALG

      public static final String HELIX_REBALANCE_ALG
      See Also:

    • ADMIN_TOPIC_REPLICATION_FACTOR

      public static final String ADMIN_TOPIC_REPLICATION_FACTOR
      The replication factor to set for admin topics.
      See Also:

    • SERVER_DISK_FULL_THRESHOLD

      public static final String SERVER_DISK_FULL_THRESHOLD
      See Also:

    • ROUTER_SINGLEGET_TARDY_LATENCY_MS

      public static final String ROUTER_SINGLEGET_TARDY_LATENCY_MS
      If a request is slower than this, it will be reported as tardy in the router metrics
      See Also:

    • ROUTER_MULTIGET_TARDY_LATENCY_MS

      public static final String ROUTER_MULTIGET_TARDY_LATENCY_MS
      See Also:

    • ROUTER_COMPUTE_TARDY_LATENCY_MS

      public static final String ROUTER_COMPUTE_TARDY_LATENCY_MS
      See Also:

    • ROUTER_ENABLE_READ_THROTTLING

      public static final String ROUTER_ENABLE_READ_THROTTLING
      See Also:

    • ROUTER_LEAKED_FUTURE_CLEANUP_POLL_INTERVAL_MS

      public static final String ROUTER_LEAKED_FUTURE_CLEANUP_POLL_INTERVAL_MS
      This config is for LeakedCompletableFutureCleanupService. Polling interval.
      See Also:

    • ROUTER_LEAKED_FUTURE_CLEANUP_THRESHOLD_MS

      public static final String ROUTER_LEAKED_FUTURE_CLEANUP_THRESHOLD_MS
      This config is for LeakedCompletableFutureCleanupService. If the CompletableFuture stays in current service beyonds the configured threshold, LeakedCompletableFutureCleanupService will complete it exceptionally.
      See Also:

    • PUSH_JOB_STATUS_STORE_CLUSTER_NAME

      public static final String PUSH_JOB_STATUS_STORE_CLUSTER_NAME
      The name of the cluster that the internal store for storing push job details records belongs to.
      See Also:

    • PUSH_JOB_GUID_MOST_SIGNIFICANT_BITS

      public static final String PUSH_JOB_GUID_MOST_SIGNIFICANT_BITS
      The most-significant-bits of the producer GUID used by VenicePushJob encoded as a long.
      See Also:

    • PUSH_JOB_GUID_LEAST_SIGNIFICANT_BITS

      public static final String PUSH_JOB_GUID_LEAST_SIGNIFICANT_BITS
      The least-significant-bits of the producer GUID used by VenicePushJob encoded as a long.
      See Also:

    • CONTROLLER_ADD_VERSION_VIA_ADMIN_PROTOCOL

      public static final String CONTROLLER_ADD_VERSION_VIA_ADMIN_PROTOCOL
      Flag to indicate whether to perform add version and start of ingestion via the admin protocol.
      See Also:

    • CONTROLLER_EARLY_DELETE_BACKUP_ENABLED

      public static final String CONTROLLER_EARLY_DELETE_BACKUP_ENABLED
      See Also:

    • PUSH_MONITOR_TYPE

      public static final String PUSH_MONITOR_TYPE
      Flag to indicate which push monitor controller will pick up for an upcoming push
      See Also:

    • PARTICIPANT_MESSAGE_STORE_ENABLED

      public static final String PARTICIPANT_MESSAGE_STORE_ENABLED
      Flag to enable the participant message store setup and write operations to the store.
      See Also:

    • CONTROLLER_SYSTEM_SCHEMA_CLUSTER_NAME

      public static final String CONTROLLER_SYSTEM_SCHEMA_CLUSTER_NAME
      The name of the cluster that should host the special stores used to serve system schemas.
      See Also:

    • SYSTEM_SCHEMA_CLUSTER_NAME

      public static final String SYSTEM_SCHEMA_CLUSTER_NAME
      The name of the cluster that should host the special stores used to serve system schemas. This config is same as CONTROLLER_SYSTEM_SCHEMA_CLUSTER_NAME, since this config will be used in various Venice Components, so we remove the `controller` prefix to avoid confusion. TODO: deprecate CONTROLLER_SYSTEM_SCHEMA_CLUSTER_NAME
      See Also:

    • ADMIN_HELIX_MESSAGING_CHANNEL_ENABLED

      public static final String ADMIN_HELIX_MESSAGING_CHANNEL_ENABLED
      Flag to enable the controller to send kill push job helix messages to the storage node upon consuming kill push job admin messages.
      See Also:

    • PARTICIPANT_MESSAGE_CONSUMPTION_DELAY_MS

      public static final String PARTICIPANT_MESSAGE_CONSUMPTION_DELAY_MS
      Minimum delay between each cycle where the storage node polls the participant message store to see if any of its ongoing push job has been killed.
      See Also:

    • ROUTER_STATEFUL_HEALTHCHECK_ENABLED

      public static final String ROUTER_STATEFUL_HEALTHCHECK_ENABLED
      See Also:

    • ROUTER_UNHEALTHY_PENDING_CONNECTION_THRESHOLD_PER_ROUTE

      public static final String ROUTER_UNHEALTHY_PENDING_CONNECTION_THRESHOLD_PER_ROUTE
      Maximum number of pending router request per storage node after which router concludes that host to be unhealthy and stops sending further request to it..
      See Also:

    • ROUTER_PENDING_CONNECTION_RESUME_THRESHOLD_PER_ROUTE

      public static final String ROUTER_PENDING_CONNECTION_RESUME_THRESHOLD_PER_ROUTE
      This is the threshold for pending request queue depth per storage node after which router resumes sending requests once a storage node which was previously marked unhealthy due to high ROUTER_UNHEALTHY_PENDING_CONNECTION_THRESHOLD_PER_ROUTE
      See Also:

    • ROUTER_PER_NODE_CLIENT_ENABLED

      public static final String ROUTER_PER_NODE_CLIENT_ENABLED
      Enables HttpAsyncClient allocation per storage node.
      See Also:

    • ROUTER_PER_NODE_CLIENT_THREAD_COUNT

      public static final String ROUTER_PER_NODE_CLIENT_THREAD_COUNT
      See Also:

    • ROUTER_HTTP2_INBOUND_ENABLED

      public static final String ROUTER_HTTP2_INBOUND_ENABLED
      Whether to support http/2 inbound request. When this feature is enabled, the Router will support both http/1.1 and http/2.
      See Also:

    • ROUTER_HTTP2_MAX_CONCURRENT_STREAMS

      public static final String ROUTER_HTTP2_MAX_CONCURRENT_STREAMS
      Indicates the maximum number of concurrent streams that the sender will allow. This limit is directional: it applies to the number of streams that the sender permits the receiver to create.
      See Also:

    • ROUTER_HTTP2_MAX_FRAME_SIZE

      public static final String ROUTER_HTTP2_MAX_FRAME_SIZE
      Indicates the size of the largest frame payload that the sender is willing to receive, in octets.
      See Also:

    • ROUTER_HTTP2_INITIAL_WINDOW_SIZE

      public static final String ROUTER_HTTP2_INITIAL_WINDOW_SIZE
      Indicates the sender's initial window size (in octets) for stream-level flow control.
      See Also:

    • ROUTER_HTTP2_HEADER_TABLE_SIZE

      public static final String ROUTER_HTTP2_HEADER_TABLE_SIZE
      Allows the sender to inform the remote endpoint of the maximum size of the header compression table used to decode header blocks, in octets.
      See Also:

    • ROUTER_HTTP2_MAX_HEADER_LIST_SIZE

      public static final String ROUTER_HTTP2_MAX_HEADER_LIST_SIZE
      This advisory setting informs a peer of the maximum size of header list that the sender is prepared to accept, in octets. The value is based on the uncompressed size of header fields, including the length of the name and value in octets plus an overhead of 32 octets for each header field.
      See Also:

    • ROUTER_HTTP_CLIENT_OPENSSL_ENABLED

      public static final String ROUTER_HTTP_CLIENT_OPENSSL_ENABLED
      Whether to enable openssl in the Router http client when talking to server.
      See Also:

    • SERVER_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS

      public static final String SERVER_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS
      In Leader/Follower state transition model, in order to avoid split brain problem (multiple leaders) as much as possible, the newly promoted leader should keep checking whether there is any new messages from the old leader in the version topic, and wait for some time (5 minutes by default) after the last message consumed before switching to leader role and potential starts producing to the version topic. Basically, the wait time could help us avoid the scenario that more than one replica is producing to the version topic.
      See Also:

    • SERVER_SYSTEM_STORE_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS

      public static final String SERVER_SYSTEM_STORE_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS
      The system store, such as replica status related requires fast leadership fail over to avoid the stable info in system store, which could affect the request routing in the read path. Since we do have a way to correct the unordered data if it really happens, such as produce a full snapshot periodically, but the freshness is very important.
      See Also:

    • SERVER_HTTP2_INBOUND_ENABLED

      public static final String SERVER_HTTP2_INBOUND_ENABLED
      Whether to support http/2 inbound request. When this feature is enabled, the server will support both http/1.1 and http/2.
      See Also:

    • SERVER_HTTP2_MAX_CONCURRENT_STREAMS

      public static final String SERVER_HTTP2_MAX_CONCURRENT_STREAMS
      Indicates the maximum number of concurrent streams that the sender will allow. This limit is directional: it applies to the number of streams that the sender permits the receiver to create.
      See Also:

    • SERVER_HTTP2_MAX_FRAME_SIZE

      public static final String SERVER_HTTP2_MAX_FRAME_SIZE
      Indicates the size of the largest frame payload that the sender is willing to receive, in octets.
      See Also:

    • SERVER_HTTP2_INITIAL_WINDOW_SIZE

      public static final String SERVER_HTTP2_INITIAL_WINDOW_SIZE
      Indicates the sender's initial window size (in octets) for stream-level flow control.
      See Also:

    • SERVER_HTTP2_HEADER_TABLE_SIZE

      public static final String SERVER_HTTP2_HEADER_TABLE_SIZE
      Allows the sender to inform the remote endpoint of the maximum size of the header compression table used to decode header blocks, in octets.
      See Also:

    • SERVER_HTTP2_MAX_HEADER_LIST_SIZE

      public static final String SERVER_HTTP2_MAX_HEADER_LIST_SIZE
      This advisory setting informs a peer of the maximum size of header list that the sender is prepared to accept, in octets. The value is based on the uncompressed size of header fields, including the length of the name and value in octets plus an overhead of 32 octets for each header field.
      See Also:

    • CONTROLLER_SSL_ENABLED

      public static final String CONTROLLER_SSL_ENABLED
      This config defines whether SSL is enabled in controller.
      See Also:

    • CONTROLLER_CLUSTER_LEADER_HAAS

      public static final String CONTROLLER_CLUSTER_LEADER_HAAS
      Flag to indicate if the controller cluster leader will be amongst one of the local Helix as a library controllers or a Helix as a service controller running remotely.
      See Also:

    • CONTROLLER_HAAS_SUPER_CLUSTER_NAME

      public static final String CONTROLLER_HAAS_SUPER_CLUSTER_NAME
      The super cluster name for HAAS. This config is required if HAAS is enabled for the creation of helix clusters.
      See Also:

    • KEY_VALUE_PROFILING_ENABLED

      public static final String KEY_VALUE_PROFILING_ENABLED
      A config that turns the key/value profiling stats on and off. This config can be placed in both Router and SNs and it is off by default. When switching it on, We will emit a fine grained histogram that reflects the distribution of key and value size. Since this will be run in the critical read path and it will emit additional ~20 stats, please be cautious when turning it on.
      See Also:

    • VENICE_STORAGE_CLUSTER_LEADER_HAAS

      public static final String VENICE_STORAGE_CLUSTER_LEADER_HAAS
      See Also:

    • PARTITIONER_CLASS

      public static final String PARTITIONER_CLASS
      A config specifies which partitioning scheme should be used by VenicePushJob.
      See Also:

    • INSTANCE_ID

      public static final String INSTANCE_ID
      A unique id that can represent this instance
      See Also:

    • ROUTER_DICTIONARY_RETRIEVAL_TIME_MS

      public static final String ROUTER_DICTIONARY_RETRIEVAL_TIME_MS
      Maximum time allowed for router to download dictionary from Storage nodes.
      See Also:

    • ROUTER_DICTIONARY_PROCESSING_THREADS

      public static final String ROUTER_DICTIONARY_PROCESSING_THREADS
      Number of threads that the Router will use to wait for dictionary to download from storage nodes and process it.
      See Also:

    • KAFKA_ADMIN_CLASS

      public static final String KAFKA_ADMIN_CLASS
      The class name to use for the PubSubAdminAdapter.
      See Also:

    • KAFKA_WRITE_ONLY_ADMIN_CLASS

      public static final String KAFKA_WRITE_ONLY_ADMIN_CLASS
      Fully-qualified class name to use for Kafka write-only admin operations.
      See Also:

    • KAFKA_READ_ONLY_ADMIN_CLASS

      public static final String KAFKA_READ_ONLY_ADMIN_CLASS
      Fully-qualified class name to use for Kafka read-only admin operations.
      See Also:

    • HELIX_HYBRID_STORE_QUOTA_ENABLED

      public static final String HELIX_HYBRID_STORE_QUOTA_ENABLED
      A config that determines whether to use Helix customized view for hybrid store quota
      See Also:

    • SERVER_SHUTDOWN_DISK_UNHEALTHY_TIME_MS

      public static final String SERVER_SHUTDOWN_DISK_UNHEALTHY_TIME_MS
      A time after which a bad SSD will trigger server shutdown.
      See Also:

    • ROUTER_EARLY_THROTTLE_ENABLED

      public static final String ROUTER_EARLY_THROTTLE_ENABLED
      Turns on early router throttling before allocating most of the router resources.
      See Also:

    • ROUTER_HEART_BEAT_ENABLED

      public static final String ROUTER_HEART_BEAT_ENABLED
      Disable router heart-beat job which marks hosts as unhealthy.
      See Also:

    • ROUTER_HTTP_CLIENT5_POOL_SIZE

      public static final String ROUTER_HTTP_CLIENT5_POOL_SIZE
      HttpClient5 pool size.
      See Also:

    • ROUTER_HTTP_CLIENT5_TOTAL_IO_THREAD_COUNT

      public static final String ROUTER_HTTP_CLIENT5_TOTAL_IO_THREAD_COUNT
      Total IO thread count for HttpClient5 pool.
      See Also:

    • ROUTER_HTTP_CLIENT5_SKIP_CIPHER_CHECK_ENABLED

      public static final String ROUTER_HTTP_CLIENT5_SKIP_CIPHER_CHECK_ENABLED
      Whether to skip the cipher check when using Httpclient5.
      See Also:

    • ROUTER_HTTPASYNCCLIENT_CLIENT_POOL_THREAD_COUNT

      public static final String ROUTER_HTTPASYNCCLIENT_CLIENT_POOL_THREAD_COUNT
      Number of IO threads used for AHAC client.
      See Also:

    • ERROR_PARTITION_AUTO_RESET_LIMIT

      public static final String ERROR_PARTITION_AUTO_RESET_LIMIT
      Maximum number of times controller will automatically reset an error partition for the current/serving version to mitigate impact of transient or long running issues during re-balance or restart.
      See Also:

    • ERROR_PARTITION_PROCESSING_CYCLE_DELAY

      public static final String ERROR_PARTITION_PROCESSING_CYCLE_DELAY
      The delay between each cycle where we iterate over all applicable resources and their partition to reset error partitions and collect data on the effectiveness of previous resets.
      See Also:

    • TERMINAL_STATE_TOPIC_CHECK_DELAY_MS

      public static final String TERMINAL_STATE_TOPIC_CHECK_DELAY_MS
      Delay between each cycle where the checker will iterate over existing topics that are yet to be truncated and poll their job status until they reach terminal state to ensure version topics in parent fabric are truncated in a timely manner.
      See Also:

    • CLIENT_USE_SYSTEM_STORE_REPOSITORY

      public static final String CLIENT_USE_SYSTEM_STORE_REPOSITORY
      A config for Da-Vinci clients to use system store based repositories or Zk based repositories.
      See Also:

    • CLIENT_SYSTEM_STORE_REPOSITORY_REFRESH_INTERVAL_SECONDS

      public static final String CLIENT_SYSTEM_STORE_REPOSITORY_REFRESH_INTERVAL_SECONDS
      The refresh interval for system store repositories that rely on periodic polling.
      See Also:

    • CONTROLLER_DISABLE_PARENT_TOPIC_TRUNCATION_UPON_COMPLETION

      public static final String CONTROLLER_DISABLE_PARENT_TOPIC_TRUNCATION_UPON_COMPLETION
      Test only config used to disable parent topic truncation upon job completion. This is needed because kafka cluster in test environment is shared between parent and child controllers. Truncating topic upon completion will confuse child controllers in certain scenarios.
      See Also:

    • D2_ZK_HOSTS_ADDRESS

      public static final String D2_ZK_HOSTS_ADDRESS
      ZooKeeper address of d2 client.
      See Also:

    • PUSH_STATUS_STORE_ENABLED

      public static final String PUSH_STATUS_STORE_ENABLED
      Config to control if push status store is enabled and should be initialized
      See Also:

    • DAVINCI_PUSH_STATUS_SCAN_ENABLED

      public static final String DAVINCI_PUSH_STATUS_SCAN_ENABLED
      See Also:

    • DAVINCI_PUSH_STATUS_SCAN_INTERVAL_IN_SECONDS

      public static final String DAVINCI_PUSH_STATUS_SCAN_INTERVAL_IN_SECONDS
      See Also:

    • DAVINCI_PUSH_STATUS_SCAN_THREAD_NUMBER

      public static final String DAVINCI_PUSH_STATUS_SCAN_THREAD_NUMBER
      See Also:

    • DAVINCI_PUSH_STATUS_SCAN_NO_REPORT_RETRY_MAX_ATTEMPTS

      public static final String DAVINCI_PUSH_STATUS_SCAN_NO_REPORT_RETRY_MAX_ATTEMPTS
      Max retry when not receiving any DaVinci status report. This is mainly for testing purpose since in local integration test, the push job runs too fast in the backend, and no DaVinci status report will mark the push job succeed right away.
      See Also:

    • DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_COUNT

      public static final String DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_COUNT
      See Also:

    • DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_RATIO

      public static final String DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_RATIO
      See Also:

    • BLOB_TRANSFER_MANAGER_ENABLED

      public static final String BLOB_TRANSFER_MANAGER_ENABLED
      See Also:

    • BLOB_TRANSFER_SNAPSHOT_RETENTION_TIME_IN_MIN

      public static final String BLOB_TRANSFER_SNAPSHOT_RETENTION_TIME_IN_MIN
      See Also:

    • BLOB_TRANSFER_MAX_CONCURRENT_SNAPSHOT_USER

      public static final String BLOB_TRANSFER_MAX_CONCURRENT_SNAPSHOT_USER
      See Also:

    • BLOB_TRANSFER_MAX_TIMEOUT_IN_MIN

      public static final String BLOB_TRANSFER_MAX_TIMEOUT_IN_MIN
      See Also:

    • BLOB_TRANSFER_DISABLED_OFFSET_LAG_THRESHOLD

      public static final String BLOB_TRANSFER_DISABLED_OFFSET_LAG_THRESHOLD
      See Also:

    • DAVINCI_P2P_BLOB_TRANSFER_SERVER_PORT

      public static final String DAVINCI_P2P_BLOB_TRANSFER_SERVER_PORT
      See Also:

    • DAVINCI_P2P_BLOB_TRANSFER_CLIENT_PORT

      public static final String DAVINCI_P2P_BLOB_TRANSFER_CLIENT_PORT
      See Also:

    • CONTROLLER_ZK_SHARED_DAVINCI_PUSH_STATUS_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED

      public static final String CONTROLLER_ZK_SHARED_DAVINCI_PUSH_STATUS_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED
      See Also:

    • PUSH_STATUS_STORE_HEARTBEAT_INTERVAL_IN_SECONDS

      public static final String PUSH_STATUS_STORE_HEARTBEAT_INTERVAL_IN_SECONDS
      Interval for Da Vinci clients to send heartbeats.
      See Also:

    • PUSH_STATUS_INSTANCE_NAME_SUFFIX

      public static final String PUSH_STATUS_INSTANCE_NAME_SUFFIX
      Config to control what's the suffix for Da Vinci instance which is reporting push status and heartbeats. By default, it is process PID if not specified, but note that PID is subject to change upon instance restart.
      See Also:

    • PUSH_STATUS_STORE_HEARTBEAT_EXPIRATION_TIME_IN_SECONDS

      public static final String PUSH_STATUS_STORE_HEARTBEAT_EXPIRATION_TIME_IN_SECONDS
      The expiration timeout. If an instance not sending heartbeats for over the expiration time, it will be considered as stale.
      See Also:

    • USE_DA_VINCI_SPECIFIC_EXECUTION_STATUS_FOR_ERROR

      public static final String USE_DA_VINCI_SPECIFIC_EXECUTION_STATUS_FOR_ERROR
      when enabled, Da Vinci Clients returns specific status codes to indicate the type of ingestion failure rather than a generic ExecutionStatus.ERROR
      See Also:

    • DAVINCI_PUSH_STATUS_CHECK_INTERVAL_IN_MS

      public static final String DAVINCI_PUSH_STATUS_CHECK_INTERVAL_IN_MS
      If the config value is non-negative, da-vinci client will batch push statues among all partitions into one single update events; DaVinciPushStatusUpdateTask will check the push status across all partitions in the same DaVinci node, and decide whether to send a status update event. This config controls the interval between each check.
      See Also:

    • ROUTER_CLIENT_SSL_HANDSHAKE_THREADS

      public static final String ROUTER_CLIENT_SSL_HANDSHAKE_THREADS
      The number of threads that will be used to perform SSL handshakes between clients and a router.
      See Also:

    • ROUTER_RESOLVE_BEFORE_SSL

      public static final String ROUTER_RESOLVE_BEFORE_SSL
      Config to control if DNS resolution should be done before SSL handshake between clients and a router. If this is enabled, the above SSL handshake thread pool will be used to perform DNS resolution, because DNS resolution before SSL and separate SSL handshake thread pool are mutually exclusive features.
      See Also:

    • ROUTER_MAX_CONCURRENT_RESOLUTIONS

      public static final String ROUTER_MAX_CONCURRENT_RESOLUTIONS
      Config to control the maximum number of concurrent DNS resolutions that can be done by the router.
      See Also:

    • ROUTER_CLIENT_RESOLUTION_RETRY_ATTEMPTS

      public static final String ROUTER_CLIENT_RESOLUTION_RETRY_ATTEMPTS
      Config to control the maximum number of attempts to resolve a client host name before giving up.
      See Also:

    • ROUTER_CLIENT_RESOLUTION_RETRY_BACKOFF_MS

      public static final String ROUTER_CLIENT_RESOLUTION_RETRY_BACKOFF_MS
      Config to control the backoff time between each resolution retry.
      See Also:

    • ROUTER_CLIENT_SSL_HANDSHAKE_QUEUE_CAPACITY

      public static final String ROUTER_CLIENT_SSL_HANDSHAKE_QUEUE_CAPACITY
      Config to control the queue capacity for the thread pool executor used for ssl handshake between clients and a router.
      See Also:

    • ROUTER_READ_QUOTA_THROTTLING_LEASE_TIMEOUT_MS

      public static final String ROUTER_READ_QUOTA_THROTTLING_LEASE_TIMEOUT_MS
      Lease timeout for leaving quota disabled for a router. If quota was disabled through an API, it will be reset after lease expiry.
      See Also:

    • CONTROLLER_SYSTEM_STORE_ACL_SYNCHRONIZATION_DELAY_MS

      public static final String CONTROLLER_SYSTEM_STORE_ACL_SYNCHRONIZATION_DELAY_MS
      The delay in ms between each synchronization attempt between the Venice store acls and its corresponding system store acls if any synchronization is needed.
      See Also:

    • LEAKED_PUSH_STATUS_CLEAN_UP_SERVICE_SLEEP_INTERVAL_MS

      public static final String LEAKED_PUSH_STATUS_CLEAN_UP_SERVICE_SLEEP_INTERVAL_MS
      This config defines the sleep interval in leaked push status clean up service.
      See Also:

    • LEAKED_RESOURCE_ALLOWED_LINGER_TIME_MS

      public static final String LEAKED_RESOURCE_ALLOWED_LINGER_TIME_MS
      This config defines the allowed linger time for a leaked resource.
      See Also:

    • CONTROLLER_DISABLE_PARENT_REQUEST_TOPIC_FOR_STREAM_PUSHES

      public static final String CONTROLLER_DISABLE_PARENT_REQUEST_TOPIC_FOR_STREAM_PUSHES
      See Also:

    • CONTROLLER_DEFAULT_READ_QUOTA_PER_ROUTER

      public static final String CONTROLLER_DEFAULT_READ_QUOTA_PER_ROUTER
      See Also:

    • LOCAL_REGION_NAME

      public static final String LOCAL_REGION_NAME
      This config will specify the region name of a controller; the region name can be customized by Venice internal.
      See Also:

    • CONTROLLER_AUTO_MATERIALIZE_META_SYSTEM_STORE

      public static final String CONTROLLER_AUTO_MATERIALIZE_META_SYSTEM_STORE
      This config controls whether to make an empty push to materialize meta system store for newly created user stores if possible.
      See Also:

    • CONTROLLER_AUTO_MATERIALIZE_DAVINCI_PUSH_STATUS_SYSTEM_STORE

      public static final String CONTROLLER_AUTO_MATERIALIZE_DAVINCI_PUSH_STATUS_SYSTEM_STORE
      See Also:

    • REPLICATION_METADATA_VERSION

      public static final String REPLICATION_METADATA_VERSION
      This will indicate which ReplicationMetadataSchemaGenerator version to use to generate replication metadata schema. This config should be set on a per-cluster level, meaning that each cluster can have its own RMD version ID.
      See Also:

    • OFFSET_LAG_DELTA_RELAX_FACTOR_FOR_FAST_ONLINE_TRANSITION_IN_RESTART

      public static final String OFFSET_LAG_DELTA_RELAX_FACTOR_FOR_FAST_ONLINE_TRANSITION_IN_RESTART
      See Also:

    • SERVER_KAFKA_CONSUMER_OFFSET_COLLECTION_ENABLED

      public static final String SERVER_KAFKA_CONSUMER_OFFSET_COLLECTION_ENABLED
      Enable offset collection for kafka topic partition from kafka consumer metrics.
      See Also:

    • SERVER_SCHEMA_PRESENCE_CHECK_ENABLED

      public static final String SERVER_SCHEMA_PRESENCE_CHECK_ENABLED
      This indicates if server will perform the schema presence check or not. By default it is set to true.
      See Also:

    • CONTROLLER_JETTY_CONFIG_OVERRIDE_PREFIX

      public static final String CONTROLLER_JETTY_CONFIG_OVERRIDE_PREFIX
      Prefix of configs to configure Jetty server in Controller.
      See Also:

    • ROUTER_META_STORE_SHADOW_READ_ENABLED

      public static final String ROUTER_META_STORE_SHADOW_READ_ENABLED
      The number of records
      See Also:

    • KAFKA_CLUSTER_MAP_KEY_NAME

      public static final String KAFKA_CLUSTER_MAP_KEY_NAME
      Defines the key names in venice.server.kafkaClustersMap
      See Also:

    • KAFKA_CLUSTER_MAP_KEY_URL

      public static final String KAFKA_CLUSTER_MAP_KEY_URL
      See Also:

    • KAFKA_CLUSTER_MAP_KEY_OTHER_URLS

      public static final String KAFKA_CLUSTER_MAP_KEY_OTHER_URLS
      See Also:

    • KAFKA_CLUSTER_MAP_SECURITY_PROTOCOL

      public static final String KAFKA_CLUSTER_MAP_SECURITY_PROTOCOL
      See Also:

    • SERVER_ENABLE_LIVE_CONFIG_BASED_KAFKA_THROTTLING

      public static final String SERVER_ENABLE_LIVE_CONFIG_BASED_KAFKA_THROTTLING
      See Also:

    • CONCURRENT_INIT_ROUTINES_ENABLED

      public static final String CONCURRENT_INIT_ROUTINES_ENABLED
      Enable the concurrent execution of the controllers' init routines, which are executed when a controller becomes a cluster leader.
      See Also:

    • SERVER_INGESTION_CHECKPOINT_DURING_GRACEFUL_SHUTDOWN_ENABLED

      public static final String SERVER_INGESTION_CHECKPOINT_DURING_GRACEFUL_SHUTDOWN_ENABLED
      A config to control graceful shutdown. True: servers will flush all remain data in producers buffers and drainer queues, and persist all data including offset metadata and producer states into disk False: servers will not flush any data during shutdown. After restart, servers will resume ingestion from the last checkpoint.
      See Also:

    • USE_PUSH_STATUS_STORE_FOR_INCREMENTAL_PUSH

      public static final String USE_PUSH_STATUS_STORE_FOR_INCREMENTAL_PUSH
      A config to control which status store to use for fetching incremental push job status from the controller. This config should be removed once the migration of push status to push status system store is complete. True: use push system status store False: use zookeeper store
      See Also:

    • META_STORE_WRITER_CLOSE_TIMEOUT_MS

      public static final String META_STORE_WRITER_CLOSE_TIMEOUT_MS
      A config to control the maximum time spent on closing the meta store writer.
      See Also:

    • META_STORE_WRITER_CLOSE_CONCURRENCY

      public static final String META_STORE_WRITER_CLOSE_CONCURRENCY
      A config to control the maximum number of concurrent meta store writer close operations.
      See Also:

    • SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_ENABLED

      public static final String SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_ENABLED
      A config to control whether VeniceServer will optimize the database for the backup version to free up memory resources occupied. TODO: explore to apply this feature to DVC as well. This feature should be very useful for RocksDB plaintable to unload the mmapped memory and it will be useful for RocksDB block-based format as well to evict the unused index/filters from the shared block cache.
      See Also:

    • SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_NO_READ_THRESHOLD_SECONDS

      public static final String SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_NO_READ_THRESHOLD_SECONDS
      A config to control the no read threshold when the database optimization should kick in.
      See Also:

    • SERVER_OPTIMIZE_DATABASE_SERVICE_SCHEDULE_INTERNAL_SECONDS

      public static final String SERVER_OPTIMIZE_DATABASE_SERVICE_SCHEDULE_INTERNAL_SECONDS
      Schedule interval for database optimization service.
      See Also:

    • UNREGISTER_METRIC_FOR_DELETED_STORE_ENABLED

      public static final String UNREGISTER_METRIC_FOR_DELETED_STORE_ENABLED
      A config that determines whether to unregister per store metrics when a store is deleted. Default is false.
      See Also:

    • FORCE_LEADER_ERROR_REPLICA_FAIL_OVER_ENABLED

      public static final String FORCE_LEADER_ERROR_REPLICA_FAIL_OVER_ENABLED
      Config to enable single leader replica disabling.
      See Also:

    • IDENTITY_PARSER_CLASS

      public static final String IDENTITY_PARSER_CLASS
      A config to specify the class to use to parse identities at runtime
      See Also:

    • VENICE_PARTITIONERS

      public static final String VENICE_PARTITIONERS
      Specifies a list of partitioners venice supported. It contains a string of concatenated partitioner class names separated by comma.
      See Also:

    • VALIDATE_VENICE_INTERNAL_SCHEMA_VERSION

      public static final String VALIDATE_VENICE_INTERNAL_SCHEMA_VERSION
      Config to check whether the protocol versions used at runtime are valid in Venice backend; if not, fail fast. Used by Samza jobs and Da Vinci clients. Default value should be true. Turn off the config in where access to routers is not feasible.
      See Also:

    • FAST_AVRO_FIELD_LIMIT_PER_METHOD

      public static final String FAST_AVRO_FIELD_LIMIT_PER_METHOD
      Config to control the maximum number of fields per method in a fast-avro generated deserializer. Can be useful if the JIT limit of 8 KB of bytecode is reached. An alternative is to use the -XX:-DontCompileHugeMethods JVM flag but that can have other side effects, so it may not be preferred.
      See Also:

    • SERVER_SSL_HANDSHAKE_THREAD_POOL_SIZE

      public static final String SERVER_SSL_HANDSHAKE_THREAD_POOL_SIZE
      Config to control the number of threads in the thread pool executor used for ssl handshake in servers. The purpose is to limit the concurrency of ssl handshakes. The feature to use a thread pool executor for handling ssl handshakes is disabled if the value of this config is <= 0. The default value is 0.
      See Also:

    • SERVER_SSL_HANDSHAKE_QUEUE_CAPACITY

      public static final String SERVER_SSL_HANDSHAKE_QUEUE_CAPACITY
      Config to control the queue capacity for the thread pool executor used for ssl handshake in servers.
      See Also:

    • CLIENT_PRODUCER_THREAD_NUM

      public static final String CLIENT_PRODUCER_THREAD_NUM
      Number of threads for online Venice producer controlling the number of concurrent write operations.
      See Also:

    • CLIENT_PRODUCER_SCHEMA_REFRESH_INTERVAL_SECONDS

      public static final String CLIENT_PRODUCER_SCHEMA_REFRESH_INTERVAL_SECONDS
      The refresh interval for online producer to refresh value schemas and update schemas that rely on periodic polling.
      See Also:

    • INGESTION_MEMORY_LIMIT

      public static final String INGESTION_MEMORY_LIMIT
      See Also:

    • INGESTION_MLOCK_ENABLED

      public static final String INGESTION_MLOCK_ENABLED
      Whether the ingestion is using mlock or not. Currently, this option is only meaningful for DaVinci use cases. Actually, this config option is being actively used, and it is a placeholder for the future optimization. The memory limit logic implemented today is assuming mlock usage, and to make it backward compatible when we want to do more optimization for non-mlock usage, we will ask the mlock user to enable this flag.
      See Also:

    • INGESTION_MEMORY_LIMIT_STORE_LIST

      public static final String INGESTION_MEMORY_LIMIT_STORE_LIST
      Only applies the memory limiter to the stores listed in this config. This is mainly used for testing purpose since ultimately, we want to enforce memory limiter against all the stores to avoid node crash. Empty config means ingestion memory limiter will apply to all the stores.
      See Also:

    • DIV_PRODUCER_STATE_MAX_AGE_MS

      public static final String DIV_PRODUCER_STATE_MAX_AGE_MS
      The maximum age (in milliseconds) of producer state retained by Data Ingestion Validation. Tuning this can prevent OOMing in cases where there is a lot of historical churn in RT producers. The age of a given producer's state is defined as: most_recent_timestamp_of_all_producers - most_recent_timestamp_of_given_producer This math is computed within a single partition, not across partitions. If enabled, the clearing of old state will happen when subscribing to a partition (e.g. on server start up), and prior to syncing progress to disk (e.g. when SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_DEFERRED_WRITE_MODE is reached). Old state clearing is disabled if this config is set to -1.
      See Also:

    • PUB_SUB_ADMIN_ADAPTER_FACTORY_CLASS

      public static final String PUB_SUB_ADMIN_ADAPTER_FACTORY_CLASS
      See Also:

    • PUB_SUB_PRODUCER_ADAPTER_FACTORY_CLASS

      public static final String PUB_SUB_PRODUCER_ADAPTER_FACTORY_CLASS
      See Also:

    • PUB_SUB_CONSUMER_ADAPTER_FACTORY_CLASS

      public static final String PUB_SUB_CONSUMER_ADAPTER_FACTORY_CLASS
      See Also:

    • PUB_SUB_SOURCE_OF_TRUTH_ADMIN_ADAPTER_FACTORY_CLASS

      public static final String PUB_SUB_SOURCE_OF_TRUTH_ADMIN_ADAPTER_FACTORY_CLASS
      Source of truth admin adapter type, mainly for avoiding topic discrepancy between multiple pub sub systems.
      See Also:

    • ROUTER_PRINCIPAL_NAME

      public static final String ROUTER_PRINCIPAL_NAME
      Venice router's principal name used for ssl. Default should contain "venice-router".
      See Also:

    • SERVER_INGESTION_HEARTBEAT_INTERVAL_MS

      public static final String SERVER_INGESTION_HEARTBEAT_INTERVAL_MS
      The time interval in milliseconds for leader replica to send heartbeat to RT topic for consumers (including the leader) to keep its latest processed upstream RT offset up-to-date in case when the RT topic ends with SOS, EOS or skipped records.
      See Also:

    • SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_ENABLED

      public static final String SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_ENABLED
      Whether to check LeaderCompleteState in the follower replica and davinci replica before marking the follower completed. This is to avoid the case that the follower replica is marked completed before the leader replica and transitions to leader if the leader replicas goes down.

      Default to false. Should be enabled only after Venice tag 0.4.154 is fully rolled out.

      See Also:

    • SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_VALID_INTERVAL_MS

      public static final String SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_VALID_INTERVAL_MS
      Follower replicas and DavinciClient will only consider heartbeats received within this time window to mark themselves as completed. This is to avoid the cases that the follower replica is marked completed based on the old heartbeat messages from a previous leader replica. Note that the leader replica keeps sending the leader completed headers in every heartbeat messages which allows the follower replica the liberty to decide based on the freshness of the heartbeat messages to avoid stale data from some edge cases scenarios.
      See Also:

    • SERVER_STUCK_CONSUMER_REPAIR_ENABLED

      public static final String SERVER_STUCK_CONSUMER_REPAIR_ENABLED
      Whether to enable stuck consumer repair in Server.
      See Also:

    • SERVER_STUCK_CONSUMER_REPAIR_INTERVAL_SECOND

      public static final String SERVER_STUCK_CONSUMER_REPAIR_INTERVAL_SECOND
      Server stuck consumer detection interval.
      See Also:

    • SERVER_STUCK_CONSUMER_REPAIR_THRESHOLD_SECOND

      public static final String SERVER_STUCK_CONSUMER_REPAIR_THRESHOLD_SECOND
      Server stuck consumer repair threshold.
      See Also:

    • SERVER_NON_EXISTING_TOPIC_INGESTION_TASK_KILL_THRESHOLD_SECOND

      public static final String SERVER_NON_EXISTING_TOPIC_INGESTION_TASK_KILL_THRESHOLD_SECOND
      When to kill the ingestion task if the topic doesn't exist for the configured period of time.
      See Also:

    • SERVER_NON_EXISTING_TOPIC_CHECK_RETRY_INTERNAL_SECOND

      public static final String SERVER_NON_EXISTING_TOPIC_CHECK_RETRY_INTERNAL_SECOND
      The config will work together with SERVER_NON_EXISTING_TOPIC_INGESTION_TASK_KILL_THRESHOLD_SECOND to decide whether a certain ingestion task should be killed or not.
      See Also:

    • SERVER_DEDICATED_CONSUMER_POOL_FOR_AA_WC_LEADER_ENABLED

      public static final String SERVER_DEDICATED_CONSUMER_POOL_FOR_AA_WC_LEADER_ENABLED
      Handling AA or WC stores is expensive because of RocksDB lookup, and this following feature will handle these writes in dedicated consumer pool, so that the full updates won't be affected by the heavy writes to these AA/WC stores.
      See Also:

    • SERVER_DEDICATED_CONSUMER_POOL_SIZE_FOR_AA_WC_LEADER

      public static final String SERVER_DEDICATED_CONSUMER_POOL_SIZE_FOR_AA_WC_LEADER
      See Also:

    • SERVER_DEDICATED_CONSUMER_POOL_SIZE_FOR_SEP_RT_LEADER

      public static final String SERVER_DEDICATED_CONSUMER_POOL_SIZE_FOR_SEP_RT_LEADER
      See Also:

    • SERVER_CONSUMER_POOL_ALLOCATION_STRATEGY

      public static final String SERVER_CONSUMER_POOL_ALLOCATION_STRATEGY
      Consumer Pool allocation strategy to rely on pool size to prioritize specific traffic. There will be 3 different types strategy supported decided by #ConsumerPoolStrategyType.
      See Also:

    • SERVER_CONSUMER_POOL_SIZE_FOR_CURRENT_VERSION_AA_WC_LEADER

      public static final String SERVER_CONSUMER_POOL_SIZE_FOR_CURRENT_VERSION_AA_WC_LEADER
      Consumer Pool for active-active or write computer leader of current version, the traffic we need to isolate due to it is more costly than normal leader processing and current version should be allocated more resources to prioritize.
      See Also:

    • SERVER_CONSUMER_POOL_SIZE_FOR_CURRENT_VERSION_SEPARATE_RT_LEADER

      public static final String SERVER_CONSUMER_POOL_SIZE_FOR_CURRENT_VERSION_SEPARATE_RT_LEADER
      Consumer Pool for separate realtime leader of current version, the traffic we need to isolate due to it is more costly than normal leader processing and current version should be allocated more resources to prioritize.
      See Also:

    • SERVER_CONSUMER_POOL_SIZE_FOR_NON_CURRENT_VERSION_AA_WC_LEADER

      public static final String SERVER_CONSUMER_POOL_SIZE_FOR_NON_CURRENT_VERSION_AA_WC_LEADER
      Consumer Pool for active-active or write computer leader of future or backup version, the traffic we need to isolate due to it is still more costly than normal leader processing and it has less priority than current version.
      See Also:

    • SERVER_CONSUMER_POOL_SIZE_FOR_CURRENT_VERSION_NON_AA_WC_LEADER

      public static final String SERVER_CONSUMER_POOL_SIZE_FOR_CURRENT_VERSION_NON_AA_WC_LEADER
      Consumer Pool for all followers, non-active-active and non-write compute leader of current version, the traffic is less costly and but it is current version with high priority.
      See Also:

    • SERVER_CONSUMER_POOL_SIZE_FOR_NON_CURRENT_VERSION_NON_AA_WC_LEADER

      public static final String SERVER_CONSUMER_POOL_SIZE_FOR_NON_CURRENT_VERSION_NON_AA_WC_LEADER
      Consumer Pool for all followers, non-active-active leader and non-write compute leader of backup and future version, the traffic is less costly and with low priority.
      See Also:

    • SERVER_RECORD_LEVEL_METRICS_WHEN_BOOTSTRAPPING_CURRENT_VERSION_ENABLED

      public static final String SERVER_RECORD_LEVEL_METRICS_WHEN_BOOTSTRAPPING_CURRENT_VERSION_ENABLED
      Whether to enable record-level metrics when bootstrapping current version. This feature will be mainly used by DaVinci to speed up bootstrapping.
      See Also:

    • CONTROLLER_DANGLING_TOPIC_CLEAN_UP_INTERVAL_SECOND

      public static final String CONTROLLER_DANGLING_TOPIC_CLEAN_UP_INTERVAL_SECOND
      Time interval for checking dangling topics between 2 different types of pub sub backends.
      See Also:

    • CONTROLLER_DANGLING_TOPIC_OCCURRENCE_THRESHOLD_FOR_CLEANUP

      public static final String CONTROLLER_DANGLING_TOPIC_OCCURRENCE_THRESHOLD_FOR_CLEANUP
      To avoid potential risk of race condition, if a topic is identified as dangling topic in number of times beyond this defined threshold, then this topic could be deleted.
      See Also:

    • PUSH_JOB_FAILURE_CHECKPOINTS_TO_DEFINE_USER_ERROR

      public static final String PUSH_JOB_FAILURE_CHECKPOINTS_TO_DEFINE_USER_ERROR
      Configure in controllers to provide a custom list of checkpoints to define user errors and to override the default checkpoint list PushJobCheckpoints.DEFAULT_PUSH_JOB_USER_ERROR_CHECKPOINTS. This is useful to emit the push job failure metrics due to user errors or not due to user errors based on the custom checkpoint list. Check PushJobCheckpoints for the list of supported checkpoints: Config should contain one or more of the checkpoints strings separated by comma. In case of invalid config, the default list of checkpoints will be used.
      See Also:

    • DEFAULT_MAX_RECORD_SIZE_BYTES

      public static final String DEFAULT_MAX_RECORD_SIZE_BYTES
      Config for the default value which is filled in when the store-level config VeniceWriter.maxRecordSizeBytes is left unset. Used as a controller config for batch push jobs. Used as a server config for nearline jobs / partial updates.
      See Also:

    • ROUTER_SINGLE_KEY_LONG_TAIL_RETRY_BUDGET_PERCENT_DECIMAL

      public static final String ROUTER_SINGLE_KEY_LONG_TAIL_RETRY_BUDGET_PERCENT_DECIMAL
      Percentage of total single get requests that are allowed for retry in decimal. e.g. 0.1 would mean up to 10% of the total single get requests are allowed for long tail retry. This is to prevent retry storm and cascading failures.
      See Also:

    • ROUTER_MULTI_KEY_LONG_TAIL_RETRY_BUDGET_PERCENT_DECIMAL

      public static final String ROUTER_MULTI_KEY_LONG_TAIL_RETRY_BUDGET_PERCENT_DECIMAL
      Percentage of total multi get requests that are allowed for retry in decimal. e.g. 0.1 would mean up to 10% of the total multi get requests are allowed for long tail retry. This is to prevent retry storm and cascading failures.
      See Also:

    • ROUTER_LONG_TAIL_RETRY_BUDGET_ENFORCEMENT_WINDOW_MS

      public static final String ROUTER_LONG_TAIL_RETRY_BUDGET_ENFORCEMENT_WINDOW_MS
      Enforcement window for router long tail retry budget token bucket. This applies to both single get and multi get retry managers.
      See Also:

    • ROUTER_RETRY_MANAGER_CORE_POOL_SIZE

      public static final String ROUTER_RETRY_MANAGER_CORE_POOL_SIZE
      The core pool size for the thread pool executor which contains threads responsible for measuring and updating all retry managers in router periodically to provide retry budget based on a percentage of the original requests.
      See Also:

    • SERVER_RESUBSCRIPTION_TRIGGERED_BY_VERSION_INGESTION_CONTEXT_CHANGE_ENABLED

      public static final String SERVER_RESUBSCRIPTION_TRIGGERED_BY_VERSION_INGESTION_CONTEXT_CHANGE_ENABLED
      Server configs to enable the topic partition re-subscription during ingestion to let bottom ingestion service aware of store version's ingestion context changed (workload type {#@link PartitionReplicaIngestionContext.WorkloadType} or {#@link VersionRole.WorkloadType} version role changed).
      See Also:

    • SERVER_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND
      Quota for AA/WC leader replica as we know AA/WC messages are expensive, so we would like to use the following throttler to limit the resource usage.
      See Also:

    • SERVER_SEP_RT_LEADER_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_SEP_RT_LEADER_QUOTA_RECORDS_PER_SECOND
      Quota for separate realtime topic leader replica as we know separate realtime topic messages are not prioritized compared to realtime topic messages, so we would like to use the following throttler to limit the resource usage.
      See Also:

    • SERVER_CURRENT_VERSION_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_CURRENT_VERSION_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND
      The following finer quota enforcement will be used when ConsumerPoolStrategyType.CURRENT_VERSION_PRIORITIZATION is enabled.
      See Also:

    • SERVER_CURRENT_VERSION_SEPARATE_RT_LEADER_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_CURRENT_VERSION_SEPARATE_RT_LEADER_QUOTA_RECORDS_PER_SECOND
      See Also:

    • SERVER_CURRENT_VERSION_NON_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_CURRENT_VERSION_NON_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND
      See Also:

    • SERVER_NON_CURRENT_VERSION_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_NON_CURRENT_VERSION_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND
      See Also:

    • SERVER_NON_CURRENT_VERSION_NON_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND

      public static final String SERVER_NON_CURRENT_VERSION_NON_AA_WC_LEADER_QUOTA_RECORDS_PER_SECOND
      See Also:

    • SERVER_CHANNEL_OPTION_WRITE_BUFFER_WATERMARK_HIGH_BYTES

      public static final String SERVER_CHANNEL_OPTION_WRITE_BUFFER_WATERMARK_HIGH_BYTES
      See Also:

    • SERVER_AA_WC_WORKLOAD_PARALLEL_PROCESSING_ENABLED

      public static final String SERVER_AA_WC_WORKLOAD_PARALLEL_PROCESSING_ENABLED
      See Also:

    • SERVER_AA_WC_WORKLOAD_PARALLEL_PROCESSING_THREAD_POOL_SIZE

      public static final String SERVER_AA_WC_WORKLOAD_PARALLEL_PROCESSING_THREAD_POOL_SIZE
      See Also:

    • SERVER_GLOBAL_RT_DIV_ENABLED

      public static final String SERVER_GLOBAL_RT_DIV_ENABLED
      See Also:

    • SERVER_NEARLINE_WORKLOAD_PRODUCER_THROUGHPUT_OPTIMIZATION_ENABLED

      public static final String SERVER_NEARLINE_WORKLOAD_PRODUCER_THROUGHPUT_OPTIMIZATION_ENABLED
      Whether to enable producer throughput optimization for realtime workload or not. Two strategies: 1. Disable compression. 2. Utilizing multiple producers per write. These two options are controlled via store-level config.
      See Also:

    • SERVER_ZSTD_DICT_COMPRESSION_LEVEL

      public static final String SERVER_ZSTD_DICT_COMPRESSION_LEVEL
      See Also:

    • SERVER_DELETE_UNASSIGNED_PARTITIONS_ON_STARTUP

      public static final String SERVER_DELETE_UNASSIGNED_PARTITIONS_ON_STARTUP
      See Also:

    • CONTROLLER_ENABLE_HYBRID_STORE_PARTITION_COUNT_UPDATE

      public static final String CONTROLLER_ENABLE_HYBRID_STORE_PARTITION_COUNT_UPDATE
      See Also:

    • PUSH_JOB_VIEW_CONFIGS

      public static final String PUSH_JOB_VIEW_CONFIGS
      See Also:

    • NAME_REPOSITORY_MAX_ENTRY_COUNT

      public static final String NAME_REPOSITORY_MAX_ENTRY_COUNT
      The maximum number of entries (per type) to be cached in the NameRepository. Default: 2000
      See Also: