Class ConfigKeys


  • public class ConfigKeys
    extends java.lang.Object
    • Field Detail

      • ZOOKEEPER_ADDRESS

        public static final java.lang.String ZOOKEEPER_ADDRESS
        See Also:
        Constant Field Values
      • ADMIN_SECURE_PORT

        public static final java.lang.String ADMIN_SECURE_PORT
        See Also:
        Constant Field Values
      • ADMIN_CHECK_READ_METHOD_FOR_KAFKA

        public static final java.lang.String ADMIN_CHECK_READ_METHOD_FOR_KAFKA
        Whether controller should check "Read" method against Kafka wildcard ACL while users request for a topic to write. By default, the config value should be true, but setting it to false would allow us to release new version of controller when the "Read" method check is not working as expected.
        See Also:
        Constant Field Values
      • PERSISTENCE_TYPE

        public static final java.lang.String PERSISTENCE_TYPE
        See Also:
        Constant Field Values
      • KAFKA_CONFIG_PREFIX

        public static final java.lang.String KAFKA_CONFIG_PREFIX
        See Also:
        Constant Field Values
      • KAFKA_BOOTSTRAP_SERVERS

        public static final java.lang.String KAFKA_BOOTSTRAP_SERVERS
        See Also:
        Constant Field Values
      • SSL_KAFKA_BOOTSTRAP_SERVERS

        public static final java.lang.String SSL_KAFKA_BOOTSTRAP_SERVERS
        See Also:
        Constant Field Values
      • KAFKA_PRODUCER_REQUEST_TIMEOUT_MS

        public static final java.lang.String KAFKA_PRODUCER_REQUEST_TIMEOUT_MS
        See Also:
        Constant Field Values
      • KAFKA_PRODUCER_RETRIES_CONFIG

        public static final java.lang.String KAFKA_PRODUCER_RETRIES_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS

        public static final java.lang.String KAFKA_PRODUCER_DELIVERY_TIMEOUT_MS
        See Also:
        Constant Field Values
      • KAFKA_CLIENT_ID_CONFIG

        public static final java.lang.String KAFKA_CLIENT_ID_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_GROUP_ID_CONFIG

        public static final java.lang.String KAFKA_GROUP_ID_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_AUTO_OFFSET_RESET_CONFIG

        public static final java.lang.String KAFKA_AUTO_OFFSET_RESET_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_ENABLE_AUTO_COMMIT_CONFIG

        public static final java.lang.String KAFKA_ENABLE_AUTO_COMMIT_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_FETCH_MIN_BYTES_CONFIG

        public static final java.lang.String KAFKA_FETCH_MIN_BYTES_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_FETCH_MAX_BYTES_CONFIG

        public static final java.lang.String KAFKA_FETCH_MAX_BYTES_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_MAX_POLL_RECORDS_CONFIG

        public static final java.lang.String KAFKA_MAX_POLL_RECORDS_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_FETCH_MAX_WAIT_MS_CONFIG

        public static final java.lang.String KAFKA_FETCH_MAX_WAIT_MS_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_MAX_PARTITION_FETCH_BYTES_CONFIG

        public static final java.lang.String KAFKA_MAX_PARTITION_FETCH_BYTES_CONFIG
        See Also:
        Constant Field Values
      • KAFKA_ADMIN_GET_TOPIC_CONFIG_MAX_RETRY_TIME_SEC

        public static final java.lang.String KAFKA_ADMIN_GET_TOPIC_CONFIG_MAX_RETRY_TIME_SEC
        See Also:
        Constant Field Values
      • KAFKA_FETCH_QUOTA_TIME_WINDOW_MS

        public static final java.lang.String KAFKA_FETCH_QUOTA_TIME_WINDOW_MS
        The time window used by the consumption throttler. Throttler will sum the requests during the time window and compare with the quota accumulated in the time window to see whether the usage exceeds quota or not.
        See Also:
        Constant Field Values
      • KAFKA_FETCH_QUOTA_BYTES_PER_SECOND

        public static final java.lang.String KAFKA_FETCH_QUOTA_BYTES_PER_SECOND
        See Also:
        Constant Field Values
      • KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND

        public static final java.lang.String KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND
        How many records that one server could consume from Kafka at most in one second. If the consume rate reached this quota, the consumption thread will be blocked until there is the available quota.
        See Also:
        Constant Field Values
      • SERVER_KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND

        public static final java.lang.String SERVER_KAFKA_FETCH_QUOTA_RECORDS_PER_SECOND
        How many records that one server could consume from Kafka at most in one second from the specified regions. If the consume rate reached this quota, the consumption thread will be blocked until there is the available quota. The value for this config is read from cluster configs in Zk.
        See Also:
        Constant Field Values
      • KAFKA_FETCH_QUOTA_UNORDERED_BYTES_PER_SECOND

        public static final java.lang.String KAFKA_FETCH_QUOTA_UNORDERED_BYTES_PER_SECOND
        See Also:
        Constant Field Values
      • KAFKA_FETCH_QUOTA_UNORDERED_RECORDS_PER_SECOND

        public static final java.lang.String KAFKA_FETCH_QUOTA_UNORDERED_RECORDS_PER_SECOND
        See Also:
        Constant Field Values
      • KAFKA_SECURITY_PROTOCOL

        public static final java.lang.String KAFKA_SECURITY_PROTOCOL
        See Also:
        Constant Field Values
      • PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_CONSUMER_POOL_SIZE

        public static final java.lang.String PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_CONSUMER_POOL_SIZE
        Number of PubSub consumer clients to be used per topic manager for fetching metadata.
        See Also:
        Constant Field Values
      • PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_THREAD_POOL_SIZE

        public static final java.lang.String PUBSUB_TOPIC_MANAGER_METADATA_FETCHER_THREAD_POOL_SIZE
        The size of the per topic manager thread pool for fetching metadata asynchronously.
        See Also:
        Constant Field Values
      • KAFKA_LOG_COMPACTION_FOR_HYBRID_STORES

        public static final java.lang.String KAFKA_LOG_COMPACTION_FOR_HYBRID_STORES
        Whether to turn on Kafka's log compaction for the store-version topics of hybrid (and real-time only) stores. Will take effect at topic creation time, and when the hybrid config for the store is turned on.
        See Also:
        Constant Field Values
      • KAFKA_MIN_LOG_COMPACTION_LAG_MS

        public static final java.lang.String KAFKA_MIN_LOG_COMPACTION_LAG_MS
        For log compaction enabled topics, this config will define the minimum time a message will remain uncompacted in the log.
        See Also:
        Constant Field Values
      • KAFKA_MIN_IN_SYNC_REPLICAS

        public static final java.lang.String KAFKA_MIN_IN_SYNC_REPLICAS
        The minimum number of in sync replicas to set for store version topics. Will use the Kafka cluster's default if not set.
        See Also:
        Constant Field Values
      • KAFKA_MIN_IN_SYNC_REPLICAS_RT_TOPICS

        public static final java.lang.String KAFKA_MIN_IN_SYNC_REPLICAS_RT_TOPICS
        The minimum number of in sync replicas to set for real-time buffer topics. Will use the Kafka cluster's default if not set.
        See Also:
        Constant Field Values
      • KAFKA_MIN_IN_SYNC_REPLICAS_ADMIN_TOPICS

        public static final java.lang.String KAFKA_MIN_IN_SYNC_REPLICAS_ADMIN_TOPICS
        The minimum number of in sync replicas to set for admin topics. Will use the Kafka cluster's default if not set.
        See Also:
        Constant Field Values
      • KAFKA_REPLICATION_FACTOR

        public static final java.lang.String KAFKA_REPLICATION_FACTOR
        The replication factor to set for store-version topics.
        See Also:
        Constant Field Values
      • KAFKA_REPLICATION_FACTOR_RT_TOPICS

        public static final java.lang.String KAFKA_REPLICATION_FACTOR_RT_TOPICS
        The replication factor to set for real-time buffer topics.
        See Also:
        Constant Field Values
      • ENABLE_NATIVE_REPLICATION_FOR_BATCH_ONLY

        public static final java.lang.String ENABLE_NATIVE_REPLICATION_FOR_BATCH_ONLY
        Cluster-level config to enable native replication for all batch-only stores.
        See Also:
        Constant Field Values
      • ENABLE_NATIVE_REPLICATION_FOR_HYBRID

        public static final java.lang.String ENABLE_NATIVE_REPLICATION_FOR_HYBRID
        Cluster-level config to enable native replication for all hybrid stores.
        See Also:
        Constant Field Values
      • ENABLE_NATIVE_REPLICATION_AS_DEFAULT_FOR_BATCH_ONLY

        public static final java.lang.String ENABLE_NATIVE_REPLICATION_AS_DEFAULT_FOR_BATCH_ONLY
        Cluster-level config to enable native replication for new batch-only stores.
        See Also:
        Constant Field Values
      • ENABLE_NATIVE_REPLICATION_AS_DEFAULT_FOR_HYBRID

        public static final java.lang.String ENABLE_NATIVE_REPLICATION_AS_DEFAULT_FOR_HYBRID
        Cluster-level config to enable native replication for new hybrid stores.
        See Also:
        Constant Field Values
      • ENABLE_ACTIVE_ACTIVE_REPLICATION_AS_DEFAULT_FOR_BATCH_ONLY_STORE

        public static final java.lang.String ENABLE_ACTIVE_ACTIVE_REPLICATION_AS_DEFAULT_FOR_BATCH_ONLY_STORE
        Cluster-level config to enable active-active replication for new batch-only stores.
        See Also:
        Constant Field Values
      • ENABLE_ACTIVE_ACTIVE_REPLICATION_AS_DEFAULT_FOR_HYBRID_STORE

        public static final java.lang.String ENABLE_ACTIVE_ACTIVE_REPLICATION_AS_DEFAULT_FOR_HYBRID_STORE
        Cluster-level config to enable active-active replication for new hybrid stores.
        See Also:
        Constant Field Values
      • ENABLE_BLOB_TRANSFER

        public static final java.lang.String ENABLE_BLOB_TRANSFER
        Store-level config to enable blob transfers (feature flag)
        See Also:
        Constant Field Values
      • CONTROLLER_SCHEMA_VALIDATION_ENABLED

        public static final java.lang.String CONTROLLER_SCHEMA_VALIDATION_ENABLED
        Sets the default for whether or not do schema validation for all stores
        See Also:
        Constant Field Values
      • DEFAULT_OFFLINE_PUSH_STRATEGY

        public static final java.lang.String DEFAULT_OFFLINE_PUSH_STRATEGY
        See Also:
        Constant Field Values
      • DEFAULT_ROUTING_STRATEGY

        public static final java.lang.String DEFAULT_ROUTING_STRATEGY
        See Also:
        Constant Field Values
      • DEFAULT_REPLICA_FACTOR

        public static final java.lang.String DEFAULT_REPLICA_FACTOR
        See Also:
        Constant Field Values
      • DEFAULT_NUMBER_OF_PARTITION

        public static final java.lang.String DEFAULT_NUMBER_OF_PARTITION
        See Also:
        Constant Field Values
      • DEFAULT_NUMBER_OF_PARTITION_FOR_HYBRID

        public static final java.lang.String DEFAULT_NUMBER_OF_PARTITION_FOR_HYBRID
        See Also:
        Constant Field Values
      • DEFAULT_MAX_NUMBER_OF_PARTITIONS

        public static final java.lang.String DEFAULT_MAX_NUMBER_OF_PARTITIONS
        See Also:
        Constant Field Values
      • DEFAULT_PARTITION_SIZE

        public static final java.lang.String DEFAULT_PARTITION_SIZE
        See Also:
        Constant Field Values
      • ENABLE_PARTITION_COUNT_ROUND_UP

        public static final java.lang.String ENABLE_PARTITION_COUNT_ROUND_UP
        Whether to round up the version-level partition count calculated by storage quota. Default is false.
        See Also:
        Constant Field Values
      • OFFLINE_JOB_START_TIMEOUT_MS

        public static final java.lang.String OFFLINE_JOB_START_TIMEOUT_MS
        See Also:
        Constant Field Values
      • DELAY_TO_REBALANCE_MS

        public static final java.lang.String DELAY_TO_REBALANCE_MS
        See Also:
        Constant Field Values
      • MIN_ACTIVE_REPLICA

        public static final java.lang.String MIN_ACTIVE_REPLICA
        See Also:
        Constant Field Values
      • CLUSTER_TO_SERVER_D2

        public static final java.lang.String CLUSTER_TO_SERVER_D2
        See Also:
        Constant Field Values
      • HELIX_SEND_MESSAGE_TIMEOUT_MS

        public static final java.lang.String HELIX_SEND_MESSAGE_TIMEOUT_MS
        See Also:
        Constant Field Values
      • REFRESH_ATTEMPTS_FOR_ZK_RECONNECT

        public static final java.lang.String REFRESH_ATTEMPTS_FOR_ZK_RECONNECT
        See Also:
        Constant Field Values
      • REFRESH_INTERVAL_FOR_ZK_RECONNECT_MS

        public static final java.lang.String REFRESH_INTERVAL_FOR_ZK_RECONNECT_MS
        See Also:
        Constant Field Values
      • KAFKA_READ_CYCLE_DELAY_MS

        public static final java.lang.String KAFKA_READ_CYCLE_DELAY_MS
        See Also:
        Constant Field Values
      • KAFKA_EMPTY_POLL_SLEEP_MS

        public static final java.lang.String KAFKA_EMPTY_POLL_SLEEP_MS
        See Also:
        Constant Field Values
      • KAFKA_FETCH_MIN_SIZE_PER_SEC

        public static final java.lang.String KAFKA_FETCH_MIN_SIZE_PER_SEC
        See Also:
        Constant Field Values
      • KAFKA_FETCH_MAX_SIZE_PER_SEC

        public static final java.lang.String KAFKA_FETCH_MAX_SIZE_PER_SEC
        See Also:
        Constant Field Values
      • KAFKA_FETCH_MAX_WAIT_TIME_MS

        public static final java.lang.String KAFKA_FETCH_MAX_WAIT_TIME_MS
        See Also:
        Constant Field Values
      • KAFKA_FETCH_PARTITION_MAX_SIZE_PER_SEC

        public static final java.lang.String KAFKA_FETCH_PARTITION_MAX_SIZE_PER_SEC
        See Also:
        Constant Field Values
      • CONTROLLER_CLUSTER_ZK_ADDRESSS

        public static final java.lang.String CONTROLLER_CLUSTER_ZK_ADDRESSS
        See Also:
        Constant Field Values
      • CONTROLLER_CLUSTER

        public static final java.lang.String CONTROLLER_CLUSTER
        Cluster name for all parent controllers
        See Also:
        Constant Field Values
      • CONTROLLER_DISABLED_ROUTES

        public static final java.lang.String CONTROLLER_DISABLED_ROUTES
        List of forbidden admin paths
        See Also:
        Constant Field Values
      • DEPRECATED_TOPIC_RETENTION_MS

        public static final java.lang.String DEPRECATED_TOPIC_RETENTION_MS
        The retention policy for deprecated topics, which includes topics for both failed jobs and retired store versions.
        See Also:
        Constant Field Values
      • DEPRECATED_TOPIC_MAX_RETENTION_MS

        public static final java.lang.String DEPRECATED_TOPIC_MAX_RETENTION_MS
        This config is to indicate the max retention policy we have setup for deprecated jobs currently and in the past. And this is used to decide whether the topic is deprecated or not during topic cleanup. The reason to have this config instead of using DEPRECATED_TOPIC_RETENTION_MS since the retention policy for deprecated jobs could change from time to time, and we need to use a max threshold to cover all the historical deprecated job topics.
        See Also:
        Constant Field Values
      • TOPIC_CLEANUP_SLEEP_INTERVAL_BETWEEN_TOPIC_LIST_FETCH_MS

        public static final java.lang.String TOPIC_CLEANUP_SLEEP_INTERVAL_BETWEEN_TOPIC_LIST_FETCH_MS
        Sleep interval between each topic list fetch from Kafka ZK in TopicCleanup service. We don't want to hit Kafka Zookeeper too frequently.
        See Also:
        Constant Field Values
      • TOPIC_CLEANUP_DELAY_FACTOR

        public static final java.lang.String TOPIC_CLEANUP_DELAY_FACTOR
        See Also:
        Constant Field Values
      • TOPIC_CLEANUP_SEND_CONCURRENT_DELETES_REQUESTS

        public static final java.lang.String TOPIC_CLEANUP_SEND_CONCURRENT_DELETES_REQUESTS
        See Also:
        Constant Field Values
      • TOPIC_DELETION_STATUS_POLL_INTERVAL_MS

        public static final java.lang.String TOPIC_DELETION_STATUS_POLL_INTERVAL_MS
        Sleep interval for polling topic deletion status from ZK.
        See Also:
        Constant Field Values
      • CONTROLLER_BACKUP_VERSION_DEFAULT_RETENTION_MS

        public static final java.lang.String CONTROLLER_BACKUP_VERSION_DEFAULT_RETENTION_MS
        The following config is to control the default retention time in milliseconds if it is not specified in store level.
        See Also:
        Constant Field Values
      • CONTROLLER_BACKUP_VERSION_RETENTION_BASED_CLEANUP_ENABLED

        public static final java.lang.String CONTROLLER_BACKUP_VERSION_RETENTION_BASED_CLEANUP_ENABLED
        The following config is to control whether to enable backup version cleanup based on retention policy or not at cluster level.
        See Also:
        Constant Field Values
      • CONTROLLER_BACKUP_VERSION_METADATA_FETCH_BASED_CLEANUP_ENABLED

        public static final java.lang.String CONTROLLER_BACKUP_VERSION_METADATA_FETCH_BASED_CLEANUP_ENABLED
        The following config is to control whether to enable backup version cleanup based on router and server current version being served.
        See Also:
        Constant Field Values
      • CONTROLLER_ZK_SHARED_META_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED

        public static final java.lang.String CONTROLLER_ZK_SHARED_META_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED
        Whether to automatically create zk shared metadata system store in Controller or not
        See Also:
        Constant Field Values
      • CONTROLLER_ENFORCE_SSL

        public static final java.lang.String CONTROLLER_ENFORCE_SSL
        Whether controller should enforce SSL.
        See Also:
        Constant Field Values
      • ADMIN_TOPIC_REMOTE_CONSUMPTION_ENABLED

        public static final java.lang.String ADMIN_TOPIC_REMOTE_CONSUMPTION_ENABLED
        Whether child controllers will directly consume the source admin topic in the parent Kafka cluster.
        See Also:
        Constant Field Values
      • ADMIN_TOPIC_SOURCE_REGION

        public static final java.lang.String ADMIN_TOPIC_SOURCE_REGION
        This config defines the source region name of the admin topic
        See Also:
        Constant Field Values
      • CHILD_CONTROLLER_ADMIN_TOPIC_CONSUMPTION_ENABLED

        public static final java.lang.String CHILD_CONTROLLER_ADMIN_TOPIC_CONSUMPTION_ENABLED
        This following config defines whether admin consumption should be enabled or not, and this config will only control the behavior in Child Controller.
        See Also:
        Constant Field Values
      • AGGREGATE_REAL_TIME_SOURCE_REGION

        public static final java.lang.String AGGREGATE_REAL_TIME_SOURCE_REGION
        This config defines the source region of aggregate hybrid store real-time data when native replication is enabled
        See Also:
        Constant Field Values
      • ALLOW_STORE_MIGRATION

        public static final java.lang.String ALLOW_STORE_MIGRATION
        Whether stores are allowed to be migrated from/to a specific cluster. The value for this config is read from cluster configs in Zk.
        See Also:
        Constant Field Values
      • ALLOW_CLUSTER_WIPE

        public static final java.lang.String ALLOW_CLUSTER_WIPE
        Whether a cluster in a data center could be wiped. Default is false.
        See Also:
        Constant Field Values
      • CONTROLLER_IN_AZURE_FABRIC

        public static final java.lang.String CONTROLLER_IN_AZURE_FABRIC
        Whether the controller is in Azure fabric. Default is false.
        See Also:
        Constant Field Values
      • CONTROLLER_STORE_GRAVEYARD_CLEANUP_ENABLED

        public static final java.lang.String CONTROLLER_STORE_GRAVEYARD_CLEANUP_ENABLED
        Whether to enable graveyard cleanup for batch-only store at cluster level. Default is false.
        See Also:
        Constant Field Values
      • CONTROLLER_STORE_GRAVEYARD_CLEANUP_DELAY_MINUTES

        public static final java.lang.String CONTROLLER_STORE_GRAVEYARD_CLEANUP_DELAY_MINUTES
        When store graveyard cleanup is enabled, delete the graveyard znode if it has not been changed for a specific time. Default is 0 min.
        See Also:
        Constant Field Values
      • CONTROLLER_STORE_GRAVEYARD_CLEANUP_SLEEP_INTERVAL_BETWEEN_LIST_FETCH_MINUTES

        public static final java.lang.String CONTROLLER_STORE_GRAVEYARD_CLEANUP_SLEEP_INTERVAL_BETWEEN_LIST_FETCH_MINUTES
        Sleep interval between each graveyard list fetch from ZK in StoreGraveyardCleanup service. Default is 15 min.
        See Also:
        Constant Field Values
      • CONTROLLER_PARENT_EXTERNAL_SUPERSET_SCHEMA_GENERATION_ENABLED

        public static final java.lang.String CONTROLLER_PARENT_EXTERNAL_SUPERSET_SCHEMA_GENERATION_ENABLED
        Whether the superset schema generation in Parent Controller should be done via passed callback or not.
        See Also:
        Constant Field Values
      • CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_SERVICE_ENABLED

        public static final java.lang.String CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_SERVICE_ENABLED
        Whether to check system store health in parent controller. Default is false.
        See Also:
        Constant Field Values
      • CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_CHECK_INTERVAL_SECONDS

        public static final java.lang.String CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_CHECK_INTERVAL_SECONDS
        Frequency to run system store health check in parent controller. Default is 30 minute (1800s).
        See Also:
        Constant Field Values
      • CONTROLLER_PARENT_SYSTEM_STORE_HEARTBEAT_CHECK_WAIT_TIME_SECONDS

        public static final java.lang.String CONTROLLER_PARENT_SYSTEM_STORE_HEARTBEAT_CHECK_WAIT_TIME_SECONDS
        The wait time before validating system store heartbeat during system store health check in parent controller. Default is 1min.
        See Also:
        Constant Field Values
      • CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_RETRY_COUNT

        public static final java.lang.String CONTROLLER_PARENT_SYSTEM_STORE_REPAIR_RETRY_COUNT
        The maximum retry count for parent controller to fix a bad system store. Default is 1.
        See Also:
        Constant Field Values
      • SYSTEM_SCHEMA_INITIALIZATION_AT_START_TIME_ENABLED

        public static final java.lang.String SYSTEM_SCHEMA_INITIALIZATION_AT_START_TIME_ENABLED
        Whether to initialize system schemas when controller starts. Default is true.
        See Also:
        Constant Field Values
      • KME_REGISTRATION_FROM_MESSAGE_HEADER_ENABLED

        public static final java.lang.String KME_REGISTRATION_FROM_MESSAGE_HEADER_ENABLED
        See Also:
        Constant Field Values
      • CONTROLLER_ENABLE_DISABLED_REPLICA_ENABLED

        public static final java.lang.String CONTROLLER_ENABLE_DISABLED_REPLICA_ENABLED
        The following config is to control whether to turn on disabled replica enabler service.
        See Also:
        Constant Field Values
      • CONTROLLER_DISABLED_REPLICA_ENABLER_INTERVAL_MS

        public static final java.lang.String CONTROLLER_DISABLED_REPLICA_ENABLER_INTERVAL_MS
        See Also:
        Constant Field Values
      • GRPC_READ_SERVER_PORT

        public static final java.lang.String GRPC_READ_SERVER_PORT
        See Also:
        Constant Field Values
      • ENABLE_GRPC_READ_SERVER

        public static final java.lang.String ENABLE_GRPC_READ_SERVER
        See Also:
        Constant Field Values
      • GRPC_SERVER_WORKER_THREAD_COUNT

        public static final java.lang.String GRPC_SERVER_WORKER_THREAD_COUNT
        See Also:
        Constant Field Values
      • LISTENER_HOSTNAME

        public static final java.lang.String LISTENER_HOSTNAME
        See Also:
        Constant Field Values
      • AUTOCREATE_DATA_PATH

        public static final java.lang.String AUTOCREATE_DATA_PATH
        See Also:
        Constant Field Values
      • ENABLE_SERVER_ALLOW_LIST

        public static final java.lang.String ENABLE_SERVER_ALLOW_LIST
        See Also:
        Constant Field Values
      • MAX_ONLINE_OFFLINE_STATE_TRANSITION_THREAD_NUMBER

        public static final java.lang.String MAX_ONLINE_OFFLINE_STATE_TRANSITION_THREAD_NUMBER
        See Also:
        Constant Field Values
      • MAX_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER

        public static final java.lang.String MAX_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER
        See Also:
        Constant Field Values
      • MAX_FUTURE_VERSION_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER

        public static final java.lang.String MAX_FUTURE_VERSION_LEADER_FOLLOWER_STATE_TRANSITION_THREAD_NUMBER
        See Also:
        Constant Field Values
      • LEADER_FOLLOWER_STATE_TRANSITION_THREAD_POOL_STRATEGY

        public static final java.lang.String LEADER_FOLLOWER_STATE_TRANSITION_THREAD_POOL_STRATEGY
        See Also:
        Constant Field Values
      • STORE_WRITER_NUMBER

        public static final java.lang.String STORE_WRITER_NUMBER
        See Also:
        Constant Field Values
      • SORTED_INPUT_DRAINER_SIZE

        public static final java.lang.String SORTED_INPUT_DRAINER_SIZE
        See Also:
        Constant Field Values
      • UNSORTED_INPUT_DRAINER_SIZE

        public static final java.lang.String UNSORTED_INPUT_DRAINER_SIZE
        See Also:
        Constant Field Values
      • STORE_WRITER_BUFFER_AFTER_LEADER_LOGIC_ENABLED

        public static final java.lang.String STORE_WRITER_BUFFER_AFTER_LEADER_LOGIC_ENABLED
        See Also:
        Constant Field Values
      • SERVER_INGESTION_TASK_MAX_IDLE_COUNT

        public static final java.lang.String SERVER_INGESTION_TASK_MAX_IDLE_COUNT
        See Also:
        Constant Field Values
      • STORE_WRITER_BUFFER_MEMORY_CAPACITY

        public static final java.lang.String STORE_WRITER_BUFFER_MEMORY_CAPACITY
        See Also:
        Constant Field Values
      • STORE_WRITER_BUFFER_NOTIFY_DELTA

        public static final java.lang.String STORE_WRITER_BUFFER_NOTIFY_DELTA
        See Also:
        Constant Field Values
      • SERVER_REST_SERVICE_STORAGE_THREAD_NUM

        public static final java.lang.String SERVER_REST_SERVICE_STORAGE_THREAD_NUM
        See Also:
        Constant Field Values
      • SERVER_NETTY_IDLE_TIME_SECONDS

        public static final java.lang.String SERVER_NETTY_IDLE_TIME_SECONDS
        See Also:
        Constant Field Values
      • SERVER_MAX_REQUEST_SIZE

        public static final java.lang.String SERVER_MAX_REQUEST_SIZE
        See Also:
        Constant Field Values
      • SERVER_SOURCE_TOPIC_OFFSET_CHECK_INTERVAL_MS

        public static final java.lang.String SERVER_SOURCE_TOPIC_OFFSET_CHECK_INTERVAL_MS
        See Also:
        Constant Field Values
      • SERVER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS

        public static final java.lang.String SERVER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS
        See Also:
        Constant Field Values
      • SERVER_NETTY_WORKER_THREADS

        public static final java.lang.String SERVER_NETTY_WORKER_THREADS
        See Also:
        Constant Field Values
      • SSL_TO_KAFKA_LEGACY

        @Deprecated
        public static final java.lang.String SSL_TO_KAFKA_LEGACY
        Deprecated.
        This config key is a misspelling. It is now considered deprecated.
        See Also:
        Constant Field Values
      • SERVER_COMPUTE_THREAD_NUM

        public static final java.lang.String SERVER_COMPUTE_THREAD_NUM
        See Also:
        Constant Field Values
      • HYBRID_QUOTA_ENFORCEMENT_ENABLED

        public static final java.lang.String HYBRID_QUOTA_ENFORCEMENT_ENABLED
        See Also:
        Constant Field Values
      • SERVER_DATABASE_MEMORY_STATS_ENABLED

        public static final java.lang.String SERVER_DATABASE_MEMORY_STATS_ENABLED
        See Also:
        Constant Field Values
      • ROUTER_MAX_READ_CAPACITY

        public static final java.lang.String ROUTER_MAX_READ_CAPACITY
        See Also:
        Constant Field Values
      • ROUTER_QUOTA_CHECK_WINDOW

        public static final java.lang.String ROUTER_QUOTA_CHECK_WINDOW
        See Also:
        Constant Field Values
      • ROUTER_USE_LOCAL_SSL_SETTINGS

        public static final java.lang.String ROUTER_USE_LOCAL_SSL_SETTINGS
        This instructs the router to start running with self signed TLS certificates as opposed to those provided via other properties. This should only be used for testing and defaults to true when running RouterServer.main().
        See Also:
        Constant Field Values
      • ROUTER_ENABLE_SSL

        public static final java.lang.String ROUTER_ENABLE_SSL
        This instructs the router to open an ssl port. This defaults to true.
        See Also:
        Constant Field Values
      • CONTROLLER_UNUSED_VALUE_SCHEMA_CLEANUP_ENABLED

        public static final java.lang.String CONTROLLER_UNUSED_VALUE_SCHEMA_CLEANUP_ENABLED
        Enable automatic removal of old unused schemas. Do not enable unless its baked enough, use admin command to delete old schemas.
        See Also:
        Constant Field Values
      • CONTROLLER_UNUSED_SCHEMA_CLEANUP_INTERVAL_SECONDS

        public static final java.lang.String CONTROLLER_UNUSED_SCHEMA_CLEANUP_INTERVAL_SECONDS
        See Also:
        Constant Field Values
      • CONTROLLER_MIN_SCHEMA_COUNT_TO_KEEP

        public static final java.lang.String CONTROLLER_MIN_SCHEMA_COUNT_TO_KEEP
        See Also:
        Constant Field Values
      • SERVER_REMOTE_INGESTION_REPAIR_SLEEP_INTERVAL_SECONDS

        public static final java.lang.String SERVER_REMOTE_INGESTION_REPAIR_SLEEP_INTERVAL_SECONDS
        See Also:
        Constant Field Values
      • SERVER_REST_SERVICE_EPOLL_ENABLED

        public static final java.lang.String SERVER_REST_SERVICE_EPOLL_ENABLED
        Whether to enable epoll in rest service layer. This will be a best-effort since epoll support is only available in Linux, not Mac.
        See Also:
        Constant Field Values
      • SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_TRANSACTIONAL_MODE

        public static final java.lang.String SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_TRANSACTIONAL_MODE
        Database sync per bytes for transactional mode. This parameter will impact the sync frequency of database after batch push. For BDB-JE transactional mode, it won't matter since BDB-JE will persist every update in the database right away; For RocksDB transactional mode, it will impact the flush frequency of memtable to SST file, and normally we would like to have this config to be comparable to the memtable size; Negative value will disable this threshold.
        See Also:
        Constant Field Values
      • SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_DEFERRED_WRITE_MODE

        public static final java.lang.String SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_DEFERRED_WRITE_MODE
        Database sync per bytes for deferred-write mode. This parameter will impact the sync frequency of database during batch push. For BDB-JE deferred-write mode, it will impact the sync frequency, but BDB-JE will do auto-flush if the memory is full; For RocksDB deferred-write mode, it will decide the file size of each SST file since every sync invocation will generate a new SST file; Negative value will disable this threshold.
        See Also:
        Constant Field Values
      • SERVER_PARTITION_GRACEFUL_DROP_DELAY_IN_SECONDS

        public static final java.lang.String SERVER_PARTITION_GRACEFUL_DROP_DELAY_IN_SECONDS
        When load balance happens, a replica could be moved to another storage node. When dropping the existing replica through Helix state transition: 'ONLINE' -> 'OFFLINE' and 'OFFLINE' -> 'DROPPED', a race condition could happen since Router in-memory partition assignment update through Zookeeper is independent from database drop in storage node, so Router could possibly forward the request to the storage node, which has just dropped the partition. To mitigate this issue, we will add a delay in state transition: 'OFFLINE' -> 'DROPPED' to drain all the incoming requests to the to-drop partition, and we will enable error-retry on Router as well.
        See Also:
        Constant Field Values
      • SERVER_LEAKED_RESOURCE_CLEAN_UP_INTERVAL_IN_MINUTES

        public static final java.lang.String SERVER_LEAKED_RESOURCE_CLEAN_UP_INTERVAL_IN_MINUTES
        When a BDB partition is dropped, the disk space is not released immediately; a checkpoint is needed to release the disk space; so a cleaner thread is spawned for the entire storage service; the cleaner thread will wake up every few hours and check whether it needs to do a checkpoint; if so, clean up each store sequentially.
        See Also:
        Constant Field Values
      • SERVER_QUOTA_ENFORCEMENT_ENABLED

        public static final java.lang.String SERVER_QUOTA_ENFORCEMENT_ENABLED
        Set to true to enable enforcement of quota by the storage node
        See Also:
        Constant Field Values
      • SEVER_CALCULATE_QUOTA_USAGE_BASED_ON_PARTITIONS_ASSIGNMENT_ENABLED

        public static final java.lang.String SEVER_CALCULATE_QUOTA_USAGE_BASED_ON_PARTITIONS_ASSIGNMENT_ENABLED
        Set to true to enable disk quota usage based on partitions assignment reported by the storage node
        See Also:
        Constant Field Values
      • SERVER_NODE_CAPACITY_RCU

        public static final java.lang.String SERVER_NODE_CAPACITY_RCU
        Number of Read Capacity Units per second that the node can handle across all stores.
        See Also:
        Constant Field Values
      • SERVER_KAFKA_MAX_POLL_RECORDS

        public static final java.lang.String SERVER_KAFKA_MAX_POLL_RECORDS
        This config is used to control the maximum records returned by every poll request. So far, Store Ingestion is throttling per poll, so if the configured value is too big, the throttling could be inaccurate and it may impact GC as well. We should try to avoid too many long-lasting objects in JVM to minimize GC overhead.
        See Also:
        Constant Field Values
      • SERVER_PUBSUB_CONSUMER_POLL_RETRY_TIMES

        public static final java.lang.String SERVER_PUBSUB_CONSUMER_POLL_RETRY_TIMES
        This config is used to control how many times PubSub consumer would retry polling during ingestion when RetriableException happens.
        See Also:
        Constant Field Values
      • SERVER_PUBSUB_CONSUMER_POLL_RETRY_BACKOFF_MS

        public static final java.lang.String SERVER_PUBSUB_CONSUMER_POLL_RETRY_BACKOFF_MS
        This config is used to control the backoff time between PubSub consumer poll retries.
        See Also:
        Constant Field Values
      • SERVER_MAX_WAIT_FOR_VERSION_INFO_MS_CONFIG

        public static final java.lang.String SERVER_MAX_WAIT_FOR_VERSION_INFO_MS_CONFIG
        Maximum duration (in milliseconds) to wait for the version information to become available in the store metadata repository before skipping Heartbeat (HB) lag monitor setup activity during state transition.
        See Also:
        Constant Field Values
      • SERVER_DISK_HEALTH_CHECK_INTERVAL_IN_SECONDS

        public static final java.lang.String SERVER_DISK_HEALTH_CHECK_INTERVAL_IN_SECONDS
        This config decides the frequency of the disk health check; the disk health check service writes 64KB data to a temporary file in the database directory and read from the file for each health check.
        See Also:
        Constant Field Values
      • SERVER_DISK_HEALTH_CHECK_TIMEOUT_IN_SECONDS

        public static final java.lang.String SERVER_DISK_HEALTH_CHECK_TIMEOUT_IN_SECONDS
        When there is an actual disk failure, health check operation would hang, so this config decides how fast the servers will start reporting unhealthy after the health check stop updating status; however, in order to reduce the possibility of false alerts (for example, the health check updates can be delayed by GC), we couldn't set the timeout too small. Currently by default, the timeout is 30 seconds.
        See Also:
        Constant Field Values
      • SERVER_DISK_HEALTH_CHECK_SERVICE_ENABLED

        public static final java.lang.String SERVER_DISK_HEALTH_CHECK_SERVICE_ENABLED
        This config is used to enable/disable the disk health check service.
        See Also:
        Constant Field Values
      • SERVER_COMPUTE_FAST_AVRO_ENABLED

        public static final java.lang.String SERVER_COMPUTE_FAST_AVRO_ENABLED
        Whether to enable fast-avro in compute request path.
        See Also:
        Constant Field Values
      • SERVER_ENABLE_PARALLEL_BATCH_GET

        public static final java.lang.String SERVER_ENABLE_PARALLEL_BATCH_GET
        Whether to enable parallel lookup for batch-get.
        See Also:
        Constant Field Values
      • SERVER_PARALLEL_BATCH_GET_CHUNK_SIZE

        public static final java.lang.String SERVER_PARALLEL_BATCH_GET_CHUNK_SIZE
        Chunk size of each task for parallel lookup of batch-get.
        See Also:
        Constant Field Values
      • SERVER_STORE_TO_EARLY_TERMINATION_THRESHOLD_MS_MAP

        public static final java.lang.String SERVER_STORE_TO_EARLY_TERMINATION_THRESHOLD_MS_MAP
        The request early termination threshold map: The key will be store name, and the value will be the actual threshold. This config is temporary, and in the long run, we will ask Venice Client to pass the actual timeout to the backend.
        See Also:
        Constant Field Values
      • SERVER_DATABASE_LOOKUP_QUEUE_CAPACITY

        public static final java.lang.String SERVER_DATABASE_LOOKUP_QUEUE_CAPACITY
        The following config is used to control the maximum database lookup requests queued, when the queue is full, server will propagate the back pressure to the caller.
        See Also:
        Constant Field Values
      • SERVER_BLOCKING_QUEUE_TYPE

        public static final java.lang.String SERVER_BLOCKING_QUEUE_TYPE
        Check the available types in com.linkedin.venice.config.BlockingQueueType
        See Also:
        Constant Field Values
      • SERVER_ROUTER_CONNECTION_WARMING_DELAY_MS

        public static final java.lang.String SERVER_ROUTER_CONNECTION_WARMING_DELAY_MS
        This config is used to control how much time Server will wait for connection warming from Routers. This is trying to avoid availability issue when router connection warming happens when Server restarts. In theory, this config should be equal to or bigger than ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_NEW_INSTANCE_DELAY_JOIN_MS.
        See Also:
        Constant Field Values
      • SERVER_CONSUMER_POOL_SIZE_PER_KAFKA_CLUSTER

        public static final java.lang.String SERVER_CONSUMER_POOL_SIZE_PER_KAFKA_CLUSTER
        Consumer pool size per Kafka cluster.
        See Also:
        Constant Field Values
      • MIN_CONSUMER_IN_CONSUMER_POOL_PER_KAFKA_CLUSTER

        public static final java.lang.String MIN_CONSUMER_IN_CONSUMER_POOL_PER_KAFKA_CLUSTER
        See Also:
        Constant Field Values
      • SERVER_SHARED_CONSUMER_ASSIGNMENT_STRATEGY

        public static final java.lang.String SERVER_SHARED_CONSUMER_ASSIGNMENT_STRATEGY
        Whether to enable partition wise balanced shared consumer assignment.
        See Also:
        Constant Field Values
      • SERVER_LEAKED_RESOURCE_CLEANUP_ENABLED

        public static final java.lang.String SERVER_LEAKED_RESOURCE_CLEANUP_ENABLED
        Whether to enable leaked resource cleanup in storage node. Right now, it only covers leaked storage partitions on disk.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_MODE

        public static final java.lang.String SERVER_INGESTION_MODE
        Ingestion mode in target storage instance. This will be applied to Da Vinci and Storage Node.
        See Also:
        Constant Field Values
      • SERVER_UNSUB_AFTER_BATCHPUSH

        public static final java.lang.String SERVER_UNSUB_AFTER_BATCHPUSH
        Unsubscribe from kakfa topic once a batch-store push finishes
        See Also:
        Constant Field Values
      • SERVER_DEDICATED_DRAINER_FOR_SORTED_INPUT_ENABLED

        public static final java.lang.String SERVER_DEDICATED_DRAINER_FOR_SORTED_INPUT_ENABLED
        Use a seprate drainer queue for sorted ingestion and un-sorted ingestion.
        See Also:
        Constant Field Values
      • INGESTION_USE_DA_VINCI_CLIENT

        public static final java.lang.String INGESTION_USE_DA_VINCI_CLIENT
        A boolean config to specify if we are using Da Vinci client for ingestion. This config will be parsed by isDaVinciConfig variable in VeniceServerConfig. By default, it is false (use Venice Server)
        See Also:
        Constant Field Values
      • SERVER_STOP_CONSUMPTION_TIMEOUT_IN_SECONDS

        public static final java.lang.String SERVER_STOP_CONSUMPTION_TIMEOUT_IN_SECONDS
        Number of retries allowed for stopConsumptionAndWait() API in StoreIngestionService.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_SERVICE_PORT

        public static final java.lang.String SERVER_INGESTION_ISOLATION_SERVICE_PORT
        Service listening port number for main ingestion service.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_APPLICATION_PORT

        public static final java.lang.String SERVER_INGESTION_ISOLATION_APPLICATION_PORT
        Service listening port number for forked ingestion process.
        See Also:
        Constant Field Values
      • SERVER_DB_READ_ONLY_FOR_BATCH_ONLY_STORE_ENABLED

        public static final java.lang.String SERVER_DB_READ_ONLY_FOR_BATCH_ONLY_STORE_ENABLED
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_STATS_CLASS_LIST

        public static final java.lang.String SERVER_INGESTION_ISOLATION_STATS_CLASS_LIST
        A list of fully-qualified class names of all stats classes that needs to be initialized in isolated ingestion process, separated by comma. This config will help isolated ingestion process to register extra stats needed for monitoring, for example: JVM GC/Memory stats. All the classes defined here will be extending AbstractVeniceStats, and will take MetricsRepository as the only parameter in their constructor.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_SSL_ENABLED

        public static final java.lang.String SERVER_INGESTION_ISOLATION_SSL_ENABLED
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_ACL_ENABLED

        public static final java.lang.String SERVER_INGESTION_ISOLATION_ACL_ENABLED
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_PRINCIPAL_NAME

        public static final java.lang.String SERVER_INGESTION_ISOLATION_PRINCIPAL_NAME
        See Also:
        Constant Field Values
      • SERVER_FORKED_PROCESS_JVM_ARGUMENT_LIST

        public static final java.lang.String SERVER_FORKED_PROCESS_JVM_ARGUMENT_LIST
        A list of JVM arguments for forked child process, separated by semicolon.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_CONNECTION_TIMEOUT_SECONDS

        public static final java.lang.String SERVER_INGESTION_ISOLATION_CONNECTION_TIMEOUT_SECONDS
        Timeout for connection between main process and forked ingestion process. If heartbeat is not refreshed within this timeout, both processes should act to reconstruct the state in order to restore connection and service.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_REQUEST_TIMEOUT_SECONDS

        public static final java.lang.String SERVER_INGESTION_ISOLATION_REQUEST_TIMEOUT_SECONDS
        Timeout for single ingestion command request sent from main process to forked ingestion process.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_HEARTBEAT_REQUEST_TIMEOUT_SECONDS

        public static final java.lang.String SERVER_INGESTION_ISOLATION_HEARTBEAT_REQUEST_TIMEOUT_SECONDS
        Timeout for single heartbeat request sent from main process to forked ingestion process.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_ISOLATION_METRIC_REQUEST_TIMEOUT_SECONDS

        public static final java.lang.String SERVER_INGESTION_ISOLATION_METRIC_REQUEST_TIMEOUT_SECONDS
        Timeout for single metric request sent from main process to forked ingestion process.
        See Also:
        Constant Field Values
      • SERVER_DATABASE_CHECKSUM_VERIFICATION_ENABLED

        public static final java.lang.String SERVER_DATABASE_CHECKSUM_VERIFICATION_ENABLED
        whether to enable checksum verification in the ingestion path from kafka to database persistency. If enabled it will keep a running checksum for all and only PUT kafka data message received in the ingestion task and periodically verify it against the key/values saved in the database persistency layer.
        See Also:
        Constant Field Values
      • SERVER_LOCAL_CONSUMER_CONFIG_PREFIX

        public static final java.lang.String SERVER_LOCAL_CONSUMER_CONFIG_PREFIX
        Any server config that start with "server.local.consumer.config.prefix" will be used as a customized consumer config for local consumer.
        See Also:
        Constant Field Values
      • SERVER_REMOTE_CONSUMER_CONFIG_PREFIX

        public static final java.lang.String SERVER_REMOTE_CONSUMER_CONFIG_PREFIX
        Any server config that start with "server.remote.consumer.config.prefix" will be used as a customized consumer config for remote consumer.
        See Also:
        Constant Field Values
      • SERVER_ROCKSDB_STORAGE_CONFIG_CHECK_ENABLED

        public static final java.lang.String SERVER_ROCKSDB_STORAGE_CONFIG_CHECK_ENABLED
        Whether to enable to check the RocksDB storage type used to open the RocksDB storage engine and how it was built. Having different storage types (BlockBasedTable and PlainTable) in read ops and write ops may lead to corruption of RocksDB storage and crash of servers.
        See Also:
        Constant Field Values
      • SERVER_SHARED_CONSUMER_NON_EXISTING_TOPIC_CLEANUP_DELAY_MS

        public static final java.lang.String SERVER_SHARED_CONSUMER_NON_EXISTING_TOPIC_CLEANUP_DELAY_MS
        This config is used to control how much time we should wait before cleaning up the corresponding ingestion task when an non-existing topic is discovered. The reason to introduce this config is that `consumer#listTopics` could only guarantee eventual consistency, so `consumer#listTopics` not returning the topic doesn't mean the topic doesn't exist in Kafka. If `consumer#listTopics` still doesn't return the topic after the configured delay, Venice SN will unsubscribe the topic, and fail the corresponding ingestion job.
        See Also:
        Constant Field Values
      • FREEZE_INGESTION_IF_READY_TO_SERVE_OR_LOCAL_DATA_EXISTS

        public static final java.lang.String FREEZE_INGESTION_IF_READY_TO_SERVE_OR_LOCAL_DATA_EXISTS
        This config will determine whether live update will be suppressed. When the feature is turned on, ingestion will stop once a partition is ready to serve; after Da Vinci client restarts or server restarts, if local data exists, ingestion will not start in Da Vinci or report ready-to-serve immediately without ingesting new data in Venice.
        See Also:
        Constant Field Values
      • KAFKA_PRODUCER_METRICS

        public static final java.lang.String KAFKA_PRODUCER_METRICS
        a comma seperated list of kafka producer metrics that will be reported. For ex. "outgoing-byte-rate,record-send-rate,batch-size-max,batch-size-avg,buffer-available-bytes,buffer-exhausted-rate"
        See Also:
        Constant Field Values
      • SERVER_DEBUG_LOGGING_ENABLED

        public static final java.lang.String SERVER_DEBUG_LOGGING_ENABLED
        Whether to print logs that are used for troubleshooting only.
        See Also:
        Constant Field Values
      • SERVER_NUM_SCHEMA_FAST_CLASS_WARMUP

        public static final java.lang.String SERVER_NUM_SCHEMA_FAST_CLASS_WARMUP
        Number of value schemas for which fast avro classes are generated for read-compute stores before push completion is reported.
        See Also:
        Constant Field Values
      • SERVER_SCHEMA_FAST_CLASS_WARMUP_TIMEOUT

        public static final java.lang.String SERVER_SCHEMA_FAST_CLASS_WARMUP_TIMEOUT
        Timeout duration of schema generation for fast class warmup period
        See Also:
        Constant Field Values
      • LOCAL_CONTROLLER_D2_SERVICE_NAME

        public static final java.lang.String LOCAL_CONTROLLER_D2_SERVICE_NAME
        See Also:
        Constant Field Values
      • LOCAL_D2_ZK_HOST

        public static final java.lang.String LOCAL_D2_ZK_HOST
        See Also:
        Constant Field Values
      • LISTENER_SSL_PORT

        public static final java.lang.String LISTENER_SSL_PORT
        See Also:
        Constant Field Values
      • HEARTBEAT_TIMEOUT

        public static final java.lang.String HEARTBEAT_TIMEOUT
        See Also:
        Constant Field Values
      • MAX_READ_CAPACITY

        public static final java.lang.String MAX_READ_CAPACITY
        See Also:
        Constant Field Values
      • SSL_TO_STORAGE_NODES

        public static final java.lang.String SSL_TO_STORAGE_NODES
        See Also:
        Constant Field Values
      • ROUTER_LONG_TAIL_RETRY_FOR_SINGLE_GET_THRESHOLD_MS

        public static final java.lang.String ROUTER_LONG_TAIL_RETRY_FOR_SINGLE_GET_THRESHOLD_MS
        After this amount of time, DDS Router will retry once for the slow storage node request. Practically, we need to manually select the threshold (e.g. P95) for retrying based on latency metrics.
        See Also:
        Constant Field Values
      • ROUTER_LONG_TAIL_RETRY_FOR_BATCH_GET_THRESHOLD_MS

        public static final java.lang.String ROUTER_LONG_TAIL_RETRY_FOR_BATCH_GET_THRESHOLD_MS
        After this amount of time, DDS Router will retry once for the slow storage node request. The configured format will be like this way: "1-10:20,11-50:50,51-200:80,201-:1000" Let me explain the config by taking one example: If the request key count for a batch-get request is '32', and it fails into this key range: [11-50], so the retry threshold for this batch-get request is 50ms. That is a limitation here: The retry threshold is actually for each scatter-gather request, but this config is not strictly with the actual key count inside each scatter-gather request, which means even if there is only one key in a scatter-gather request with the above example, Router will wait for 50ms to retry this scatter-gather request. For now, it is not big issue since for now we mostly want to use this config to skip the storage node, which is experiencing long GC pause. So coarse-grained config should be good enough.
        See Also:
        Constant Field Values
      • ROUTER_SMART_LONG_TAIL_RETRY_ENABLED

        public static final java.lang.String ROUTER_SMART_LONG_TAIL_RETRY_ENABLED
        Whether to enable smart long tail retry logic, and this logic is only useful for batch-get retry currently. This feature is used to avoid the unnecessary retries in the following scenarios: 1. Router is suffering long GC pause, no matter whether Storage Node is fast or not; 2. The retried Storage Node is slow according to the original request; For case 1, unnecessary retries will make Router GC behavior even worse; For case 2, unnecessary retries to the slow Storage Node will make the slow Storage Node even slower, and the overall latency won't be improved; For case 1, here is how smart retry works: 1. When the delay between the retry request and the original request is over ROUTER_LONG_TAIL_RETRY_FOR_BATCH_GET_THRESHOLD_MS + ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS, smart retry logic will treat the current Router to be in bad state (long GC pause or too busy), the retry request will be aborted; 2.ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS is the way to measure whether Router is in good state or not, and need to be tuned in prod; For case 2, the retry request will be aborted if the original request to the same storage node hasn't returned, and the slowness measurement is inside one request when scatter-gathering.
        See Also:
        Constant Field Values
      • ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS

        public static final java.lang.String ROUTER_SMART_LONG_TAIL_RETRY_ABORT_THRESHOLD_MS
        This config is used to tune the smart long-tail retry logic to avoid unnecessary retries, check more details: ROUTER_SMART_LONG_TAIL_RETRY_ENABLED
        See Also:
        Constant Field Values
      • ROUTER_LONG_TAIL_RETRY_MAX_ROUTE_FOR_MULTI_KEYS_REQ

        public static final java.lang.String ROUTER_LONG_TAIL_RETRY_MAX_ROUTE_FOR_MULTI_KEYS_REQ
        This config is used to limit the maximum retries in route unit. In large batch-get/compute cluster, when enabling long-tail retry, in the worst scenarios, Router could trigger a retry storm since each route could retry independently. This config is used to specify the maximum retry in route unit. If the configured value is 1, it means the current request will at most one route. This could mitigate the latency issue in most of the case since the chance to have multiple slow storage nodes is low, also even with unlimited retries, it won't help since multiple replicas for the same partition are in a degraded state.
        See Also:
        Constant Field Values
      • ROUTER_MAX_KEY_COUNT_IN_MULTIGET_REQ

        public static final java.lang.String ROUTER_MAX_KEY_COUNT_IN_MULTIGET_REQ
        The max key count allowed in one multi-get request. For now, it is configured in host level, and we could consider to configure it in store level.
        See Also:
        Constant Field Values
      • ROUTER_CONNECTION_LIMIT

        public static final java.lang.String ROUTER_CONNECTION_LIMIT
        See Also:
        Constant Field Values
      • ROUTER_HTTP_CLIENT_POOL_SIZE

        public static final java.lang.String ROUTER_HTTP_CLIENT_POOL_SIZE
        The http client pool size being used in one Router;
        See Also:
        Constant Field Values
      • ROUTER_IO_WORKER_COUNT

        public static final java.lang.String ROUTER_IO_WORKER_COUNT
        Config to control the Netty IO thread count for the Router Server
        See Also:
        Constant Field Values
      • ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE

        public static final java.lang.String ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE
        The max connection number per route (to one storage node);
        See Also:
        Constant Field Values
      • ROUTER_MAX_OUTGOING_CONNECTION

        public static final java.lang.String ROUTER_MAX_OUTGOING_CONNECTION
        The max connection number in one Router to storage nodes;
        See Also:
        Constant Field Values
      • ROUTER_MAX_PENDING_REQUEST

        public static final java.lang.String ROUTER_MAX_PENDING_REQUEST
        This config is used to bound the pending request. Without this config, the accumulated requests in Http Async Client could grow unlimitedly, which would put Router in a non-recoverable state because of long GC pause introduced by the increasing memory usage. If the incoming request exceeds this configured threshold, Router will return 503 (Service Unavailable).
        See Also:
        Constant Field Values
      • ROUTER_MULTI_KEY_ROUTING_STRATEGY

        public static final java.lang.String ROUTER_MULTI_KEY_ROUTING_STRATEGY
        This config is used to define the routing strategy for multi-key requests. Please check VeniceMultiKeyRoutingStrategy to find available routing strategy.
        See Also:
        Constant Field Values
      • ROUTER_HELIX_ASSISTED_ROUTING_GROUP_SELECTION_STRATEGY

        public static final java.lang.String ROUTER_HELIX_ASSISTED_ROUTING_GROUP_SELECTION_STRATEGY
        Helix group selection strategy when Helix assisted routing is enabled. Available strategies listed here: HelixGroupSelectionStrategyEnum.
        See Also:
        Constant Field Values
      • ROUTER_PER_STORE_ROUTER_QUOTA_BUFFER

        public static final java.lang.String ROUTER_PER_STORE_ROUTER_QUOTA_BUFFER
        See Also:
        Constant Field Values
      • ROUTER_DNS_CACHE_ENABLED

        public static final java.lang.String ROUTER_DNS_CACHE_ENABLED
        Whether to enable customized dns cache in router or not. This is mostly to address slow DNS lookup issue.
        See Also:
        Constant Field Values
      • ROUTE_DNS_CACHE_HOST_PATTERN

        public static final java.lang.String ROUTE_DNS_CACHE_HOST_PATTERN
        The host matching the configured host pattern will be cached if ROUTER_DNS_CACHE_ENABLED is true.
        See Also:
        Constant Field Values
      • ROUTER_STORAGE_NODE_CLIENT_TYPE

        public static final java.lang.String ROUTER_STORAGE_NODE_CLIENT_TYPE
        Whether the router use netty http client or apache http async client
        See Also:
        Constant Field Values
      • ROUTER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS

        public static final java.lang.String ROUTER_NETTY_GRACEFUL_SHUTDOWN_PERIOD_SECONDS
        Netty graceful shutdown period considering the following factors: 1. D2 de-announcement could take some time; 2. Client could take some time to receive/apply the zk update event from D2 server about router shutdown; 3. Router needs some time to handle already-received client requests;
        See Also:
        Constant Field Values
      • ROUTER_CLIENT_DECOMPRESSION_ENABLED

        public static final java.lang.String ROUTER_CLIENT_DECOMPRESSION_ENABLED
        See Also:
        Constant Field Values
      • ROUTER_COMPUTE_FAST_AVRO_ENABLED

        public static final java.lang.String ROUTER_COMPUTE_FAST_AVRO_ENABLED
        Whether to enable fast-avro in router;
        See Also:
        Constant Field Values
      • ROUTER_SOCKET_TIMEOUT

        public static final java.lang.String ROUTER_SOCKET_TIMEOUT
        Socket timeout config for the connection manager from router to server
        See Also:
        Constant Field Values
      • ROUTER_CONNECTION_TIMEOUT

        public static final java.lang.String ROUTER_CONNECTION_TIMEOUT
        Timeout for building a new connection from router to server
        See Also:
        Constant Field Values
      • ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_ENABLED

        public static final java.lang.String ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_ENABLED
        Whether to enable the cleanup of the idle connections to storage node. Recently, we are seeing latency spike because of new connection setup, and we hope the total available connections will be more stable by disabling the idle connection cleanup. The potential long-term solutions could be connection warm-up for HTTP/1.1 or adopting HTTP/2
        See Also:
        Constant Field Values
      • ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_THRESHOLD_MINS

        public static final java.lang.String ROUTER_IDLE_CONNECTION_TO_SERVER_CLEANUP_THRESHOLD_MINS
        The idle threshold for cleaning up the connections to storage node.
        See Also:
        Constant Field Values
      • ROUTER_FULL_PENDING_QUEUE_SERVER_OOR_MS

        public static final java.lang.String ROUTER_FULL_PENDING_QUEUE_SERVER_OOR_MS
        The following config controls how long the server with full pending queue will be taken OOR.
        See Also:
        Constant Field Values
      • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_ENABLED

        public static final java.lang.String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_ENABLED
        Connection warming feature for httpasynclient. So far, it only works when Router starts and runs in http-client-per-route mode, and it will try to warm up ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE connections per route.
        See Also:
        Constant Field Values
      • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_SLEEP_INTERVAL_MS

        public static final java.lang.String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_SLEEP_INTERVAL_MS
        When Router starts, for a given route, the following config controls the warming up speed to minimize the impact to storage nodes.
        See Also:
        Constant Field Values
      • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_LOW_WATER_MARK

        public static final java.lang.String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_LOW_WATER_MARK
        When the available connections in an httpasyncclient is below the low water mark, the connection warming service will try to spin up a new client to replace it. In theory, this config must be lower than ROUTER_MAX_OUTGOING_CONNECTION_PER_ROUTE.
        See Also:
        Constant Field Values
      • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_EXECUTOR_THREAD_NUM

        public static final java.lang.String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_EXECUTOR_THREAD_NUM
        Connection warming executor thread num.
        See Also:
        Constant Field Values
      • ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_NEW_INSTANCE_DELAY_JOIN_MS

        public static final java.lang.String ROUTER_HTTPASYNCCLIENT_CONNECTION_WARMING_NEW_INSTANCE_DELAY_JOIN_MS
        For the new instance (Storage Node) detected by Router, the following config defines how much delay because of connection warming it could tolerate. If the connection warming takes longer than it, Router will put it in to serve online traffic by creating a new client without connection warming.
        See Also:
        Constant Field Values
      • ROUTER_HTTPAYSNCCLIENT_CONNECTION_WARMING_SOCKET_TIMEOUT_MS

        public static final java.lang.String ROUTER_HTTPAYSNCCLIENT_CONNECTION_WARMING_SOCKET_TIMEOUT_MS
        This config is used to control the socket timeout for connection warming requests. In some cases, we would like to have different(maybe longer timeout) than the regular requests considering the deployment procedure, and the connection warming requests could be very instensive.
        See Also:
        Constant Field Values
      • ROUTER_ASYNC_START_ENABLED

        public static final java.lang.String ROUTER_ASYNC_START_ENABLED
        Whether to enable async start in Router startup procedure. The reason to introduce this feature is that in some env, the dependent services could be started out of order. IMPORTANT: enabling this feature won't guarantee that a successful restarted Router will be in a healthy state, since async start will exit later if it detects any errors.
        See Also:
        Constant Field Values
      • CONTROLLER_CLUSTER_REPLICA

        public static final java.lang.String CONTROLLER_CLUSTER_REPLICA
        Venice uses a helix cluster to assign controllers to each named venice cluster. This is the number of controllers assigned to each venice cluster. Should normally be 3; one leader controller and 2 standby controllers.
        See Also:
        Constant Field Values
      • TOPIC_CREATION_THROTTLING_TIME_WINDOW_MS

        public static final java.lang.String TOPIC_CREATION_THROTTLING_TIME_WINDOW_MS
        The time window in ms used to throttle the Kafka topic creation, during the time window, only 1 topic is allowed to be created.
        See Also:
        Constant Field Values
      • TOPIC_MANAGER_KAFKA_OPERATION_TIMEOUT_MS

        public static final java.lang.String TOPIC_MANAGER_KAFKA_OPERATION_TIMEOUT_MS
        Timeout for create topic and delete topic operations.
        See Also:
        Constant Field Values
      • MIN_NUMBER_OF_UNUSED_KAFKA_TOPICS_TO_PRESERVE

        public static final java.lang.String MIN_NUMBER_OF_UNUSED_KAFKA_TOPICS_TO_PRESERVE
        This is the minimum number of Kafka topics that are guaranteed to be preserved by the leaky topic clean up routine. The topics with the highest version numbers will be favored by this preservative behavior. All other topics (i.e.: those with smaller version numbers) which Venice does not otherwise know about from its metadata will be considered leaked resources and thus be eligible for clean up. A value greater than zero is recommended for Storage Node to not run into UNKNOWN_TOPIC_OR_PARTITION issues N.B.: A known limitation of this preservation setting is that during store deletion, if a topic has been leaked recently due to an aborted push, then there is an edge case where that topic may leak forever. This leak does not happen if the latest store-versions are successful pushes, rather than failed ones. Furthermore, if a store with the same name is ever re-created, then the clean up routine would resume and clean up the older leaky topics successfully. This edge case is deemed a small enough concern for now, though it could be addressed with a more significant redesign of the replication pipeline.
        See Also:
        Constant Field Values
      • MIN_NUMBER_OF_STORE_VERSIONS_TO_PRESERVE

        public static final java.lang.String MIN_NUMBER_OF_STORE_VERSIONS_TO_PRESERVE
        This is the number of fully-functional store-versions we wish to maintain. All resources of these versions will be preserved (Helix resource, Storage Node data, Kafka topic, replication streams, etc.), and a swap to these versions should be possible at all times. This setting must be set to 1 or greater.
        See Also:
        Constant Field Values
      • CONTROLLER_PARENT_MODE

        public static final java.lang.String CONTROLLER_PARENT_MODE
        Whether current controller is parent or not
        See Also:
        Constant Field Values
      • PARENT_CONTROLLER_MAX_ERRORED_TOPIC_NUM_TO_KEEP

        public static final java.lang.String PARENT_CONTROLLER_MAX_ERRORED_TOPIC_NUM_TO_KEEP
        This config is used to control how many errored topics we are going to keep in parent cluster. This is mostly used to investigate the Kafka missing message issue. If the issue gets resolved, we could change this config to be '0'.
        See Also:
        Constant Field Values
      • CHILD_CLUSTER_URL_PREFIX

        public static final java.lang.String CHILD_CLUSTER_URL_PREFIX
        Only required when controller.parent.mode=true This prefix specifies the location of every child cluster that is being fed by this parent cluster. The format for key/value would be like "key=child.cluster.url.dc-0, value=url1;url2;url3" the cluster name should be human-readable, ex: dc-0 the url should be of the form http://host:port Note that every cluster name supplied must also be specified in the child.cluster.allowlist in order to be included
        See Also:
        Constant Field Values
      • CHILD_DATA_CENTER_KAFKA_URL_PREFIX

        public static final java.lang.String CHILD_DATA_CENTER_KAFKA_URL_PREFIX
        Config prefix for Kafka bootstrap url in all child fabrics; parent controllers need to know the Kafka url in all fabrics for native replication.
        See Also:
        Constant Field Values
      • CHILD_CLUSTER_D2_SERVICE_NAME

        public static final java.lang.String CHILD_CLUSTER_D2_SERVICE_NAME
        D2 Service name for the child controllers in local datacenter
        See Also:
        Constant Field Values
      • CLUSTER_DISCOVERY_D2_SERVICE

        public static final java.lang.String CLUSTER_DISCOVERY_D2_SERVICE
        D2 Service name for cluster discovery
        See Also:
        Constant Field Values
      • NATIVE_REPLICATION_SOURCE_FABRIC

        public static final java.lang.String NATIVE_REPLICATION_SOURCE_FABRIC
        The default source fabric used for native replication
        See Also:
        Constant Field Values
      • NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_BATCH_ONLY_STORES

        public static final java.lang.String NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_BATCH_ONLY_STORES
        The default source fabric used for native replication for batch only stores.
        See Also:
        Constant Field Values
      • NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_HYBRID_STORES

        public static final java.lang.String NATIVE_REPLICATION_SOURCE_FABRIC_AS_DEFAULT_FOR_HYBRID_STORES
        The default source fabric used for native replication for hybrid stores.
        See Also:
        Constant Field Values
      • ENABLE_INCREMENTAL_PUSH_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES

        public static final java.lang.String ENABLE_INCREMENTAL_PUSH_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES
        We will use this config to determine whether we should enable incremental push for hybrid active-active user stores. If this config is set to true, we will enable incremental push for hybrid active-active user stores.
        See Also:
        Constant Field Values
      • ENABLE_PARTIAL_UPDATE_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES

        public static final java.lang.String ENABLE_PARTIAL_UPDATE_FOR_HYBRID_ACTIVE_ACTIVE_USER_STORES
        We will use this config to determine whether we should enable partial update for hybrid active-active user stores. If this config is set to true, we will enable partial update for hybrid active-active user stores whose latest value schema meets partial update feature requirement.
        See Also:
        Constant Field Values
      • ENABLE_PARTIAL_UPDATE_FOR_HYBRID_NON_ACTIVE_ACTIVE_USER_STORES

        public static final java.lang.String ENABLE_PARTIAL_UPDATE_FOR_HYBRID_NON_ACTIVE_ACTIVE_USER_STORES
        We will use this config to determine whether we should enable partial update for hybrid non-active-active user stores. If this config is set to true, we will enable partial update for hybrid active-active user stores whose latest value schema meets partial update feature requirement.
        See Also:
        Constant Field Values
      • EMERGENCY_SOURCE_REGION

        public static final java.lang.String EMERGENCY_SOURCE_REGION
        The highest priority source fabric selection config, specified in parent controller.
        See Also:
        Constant Field Values
      • CHILD_CLUSTER_WHITELIST

        @Deprecated
        public static final java.lang.String CHILD_CLUSTER_WHITELIST
        Deprecated.
        See Also:
        Constant Field Values
      • CHILD_CLUSTER_ALLOWLIST

        public static final java.lang.String CHILD_CLUSTER_ALLOWLIST
        Only required when controller.parent.mode=true This is a comma-separated allowlist of cluster names used in the keys with the child.cluster.url prefix. Example, if we have the following child.cluster.url keys: child.cluster.url.cluster1=... child.cluster.url.cluster2=... child.cluster.url.cluster3=... And we want to use all three cluster, then we set child.cluster.allowlist=cluster1,cluster2,cluster3 If we only want to use clusters 1 and 3 we can set child.cluster.allowlist=cluster1,cluster3
        See Also:
        Constant Field Values
      • NATIVE_REPLICATION_FABRIC_WHITELIST

        @Deprecated
        public static final java.lang.String NATIVE_REPLICATION_FABRIC_WHITELIST
        Deprecated.
        See Also:
        Constant Field Values
      • NATIVE_REPLICATION_FABRIC_ALLOWLIST

        public static final java.lang.String NATIVE_REPLICATION_FABRIC_ALLOWLIST
        Previously CHILD_CLUSTER_ALLOWLIST is used to also represent the allowlist of source fabrics for native replication; however, the final migration plan decides that a Kafka cluster in parent fabric can also be the source fabric, so the below config is introduced to represent all potential source fabrics for native replication.
        See Also:
        Constant Field Values
      • PARENT_KAFKA_CLUSTER_FABRIC_LIST

        public static final java.lang.String PARENT_KAFKA_CLUSTER_FABRIC_LIST
        A list of potential parent fabrics. Logically, there is only one parent fabric; during native replication migration, there will be two Kafka clusters in parent fabric, so we need two fabric names to represent the two different Kafka cluster url. TODO: deprecate this config after native replication migration is complete.
        See Also:
        Constant Field Values
      • ACTIVE_ACTIVE_REAL_TIME_SOURCE_FABRIC_LIST

        public static final java.lang.String ACTIVE_ACTIVE_REAL_TIME_SOURCE_FABRIC_LIST
        A list of fabrics that are source(s) of the active active real time replication. When active-active replication is enabled on the controller ACTIVE_ACTIVE_ENABLED_ON_CONTROLLER is true, this list should contain fabrics where the Venice server should consume from when it accepts the TS (TopicSwitch) message. Example value of this config: "dc-0, dc-1, dc-2".
        See Also:
        Constant Field Values
      • PARENT_CONTROLLER_WAITING_TIME_FOR_CONSUMPTION_MS

        public static final java.lang.String PARENT_CONTROLLER_WAITING_TIME_FOR_CONSUMPTION_MS
        When the parent controller receives an admin write operation, it replicates that message to the admin kafka stream. After replication the parent controller consumes the message from the stream and processes it there. This is the timeout for waiting until that consumption happens.
        See Also:
        Constant Field Values
      • ADMIN_CONSUMPTION_TIMEOUT_MINUTES

        public static final java.lang.String ADMIN_CONSUMPTION_TIMEOUT_MINUTES
        If there is a failure in consuming from the admin topic, skip the message after retrying for this many minutes Default 5 days
        See Also:
        Constant Field Values
      • ADMIN_CONSUMPTION_CYCLE_TIMEOUT_MS

        public static final java.lang.String ADMIN_CONSUMPTION_CYCLE_TIMEOUT_MS
        The maximum time allowed for worker threads to execute admin messages in one cycle. A cycle is the processing of delegated admin messages by some number of worker thread(s) defined by ADMIN_CONSUMPTION_MAX_WORKER_THREAD_POOL_SIZE. Each worker thread will try to empty the queue for a store before moving on to process admin messages for another store. The cycle is completed either by finishing all delegated admin messages or timing out with this config. TODO: Note that the timeout is for all stores in the cycle and not individual stores. Meaning that some stores may starve.
        See Also:
        Constant Field Values
      • ADMIN_CONSUMPTION_MAX_WORKER_THREAD_POOL_SIZE

        public static final java.lang.String ADMIN_CONSUMPTION_MAX_WORKER_THREAD_POOL_SIZE
        The maximum number of threads allowed in the pool for executing admin messages.
        See Also:
        Constant Field Values
      • STORAGE_ENGINE_OVERHEAD_RATIO

        public static final java.lang.String STORAGE_ENGINE_OVERHEAD_RATIO
        This factor is used to estimate potential push size. VPJ reducer multiplies it with total record size and compares it with store storage quota TODO: it will be moved to Store metadata if we allow stores have various storage engine types.
        See Also:
        Constant Field Values
      • ENABLE_OFFLINE_PUSH_SSL_WHITELIST

        @Deprecated
        public static final java.lang.String ENABLE_OFFLINE_PUSH_SSL_WHITELIST
        Deprecated.
        See Also:
        Constant Field Values
      • ENABLE_OFFLINE_PUSH_SSL_ALLOWLIST

        public static final java.lang.String ENABLE_OFFLINE_PUSH_SSL_ALLOWLIST
        The switcher to enable/disable the allowlist of ssl offline pushes. If we disable the allowlist here, depends on the config "SSL_TO_KAFKA", all pushes will be secured by SSL or none of pushes will be secured by SSL.
        See Also:
        Constant Field Values
      • ENABLE_HYBRID_PUSH_SSL_WHITELIST

        @Deprecated
        public static final java.lang.String ENABLE_HYBRID_PUSH_SSL_WHITELIST
        Deprecated.
        See Also:
        Constant Field Values
      • ENABLE_HYBRID_PUSH_SSL_ALLOWLIST

        public static final java.lang.String ENABLE_HYBRID_PUSH_SSL_ALLOWLIST
        The switcher to enable/disable the allowlist of ssl hybrid pushes including both batch and near-line pushes for that store. If we disable the allowlist here, depends on the config "SSL_TO_KAFKA", all pushes will be secured by SSL or none of pushes will be secured by SSL.
        See Also:
        Constant Field Values
      • PUSH_SSL_WHITELIST

        @Deprecated
        public static final java.lang.String PUSH_SSL_WHITELIST
        Deprecated.
        See Also:
        Constant Field Values
      • PUSH_SSL_ALLOWLIST

        public static final java.lang.String PUSH_SSL_ALLOWLIST
        Allowlist of stores which are allowed to push data with SSL.
        See Also:
        Constant Field Values
      • ENFORCE_SECURE_ROUTER

        public static final java.lang.String ENFORCE_SECURE_ROUTER
        Whether to block storage requests on the non-ssl port. Will still allow metadata requests on the non-ssl port and will log storage requests on the non-ssl port even if set to false;
        See Also:
        Constant Field Values
      • HELIX_REBALANCE_ALG

        public static final java.lang.String HELIX_REBALANCE_ALG
        See Also:
        Constant Field Values
      • ADMIN_TOPIC_REPLICATION_FACTOR

        public static final java.lang.String ADMIN_TOPIC_REPLICATION_FACTOR
        The replication factor to set for admin topics.
        See Also:
        Constant Field Values
      • SERVER_DISK_FULL_THRESHOLD

        public static final java.lang.String SERVER_DISK_FULL_THRESHOLD
        See Also:
        Constant Field Values
      • ROUTER_SINGLEGET_TARDY_LATENCY_MS

        public static final java.lang.String ROUTER_SINGLEGET_TARDY_LATENCY_MS
        If a request is slower than this, it will be reported as tardy in the router metrics
        See Also:
        Constant Field Values
      • ROUTER_MULTIGET_TARDY_LATENCY_MS

        public static final java.lang.String ROUTER_MULTIGET_TARDY_LATENCY_MS
        See Also:
        Constant Field Values
      • ROUTER_COMPUTE_TARDY_LATENCY_MS

        public static final java.lang.String ROUTER_COMPUTE_TARDY_LATENCY_MS
        See Also:
        Constant Field Values
      • ROUTER_ENABLE_READ_THROTTLING

        public static final java.lang.String ROUTER_ENABLE_READ_THROTTLING
        See Also:
        Constant Field Values
      • ROUTER_LEAKED_FUTURE_CLEANUP_POLL_INTERVAL_MS

        public static final java.lang.String ROUTER_LEAKED_FUTURE_CLEANUP_POLL_INTERVAL_MS
        This config is for LeakedCompletableFutureCleanupService. Polling interval.
        See Also:
        Constant Field Values
      • ROUTER_LEAKED_FUTURE_CLEANUP_THRESHOLD_MS

        public static final java.lang.String ROUTER_LEAKED_FUTURE_CLEANUP_THRESHOLD_MS
        This config is for LeakedCompletableFutureCleanupService. If the CompletableFuture stays in current service beyonds the configured threshold, LeakedCompletableFutureCleanupService will complete it exceptionally.
        See Also:
        Constant Field Values
      • PUSH_JOB_STATUS_STORE_CLUSTER_NAME

        public static final java.lang.String PUSH_JOB_STATUS_STORE_CLUSTER_NAME
        The name of the cluster that the internal store for storing push job details records belongs to.
        See Also:
        Constant Field Values
      • PUSH_JOB_GUID_MOST_SIGNIFICANT_BITS

        public static final java.lang.String PUSH_JOB_GUID_MOST_SIGNIFICANT_BITS
        The most-significant-bits of the producer GUID used by VenicePushJob encoded as a long.
        See Also:
        Constant Field Values
      • PUSH_JOB_GUID_LEAST_SIGNIFICANT_BITS

        public static final java.lang.String PUSH_JOB_GUID_LEAST_SIGNIFICANT_BITS
        The least-significant-bits of the producer GUID used by VenicePushJob encoded as a long.
        See Also:
        Constant Field Values
      • CONTROLLER_ADD_VERSION_VIA_ADMIN_PROTOCOL

        public static final java.lang.String CONTROLLER_ADD_VERSION_VIA_ADMIN_PROTOCOL
        Flag to indicate whether to perform add version and start of ingestion via the admin protocol.
        See Also:
        Constant Field Values
      • CONTROLLER_EARLY_DELETE_BACKUP_ENABLED

        public static final java.lang.String CONTROLLER_EARLY_DELETE_BACKUP_ENABLED
        See Also:
        Constant Field Values
      • PUSH_MONITOR_TYPE

        public static final java.lang.String PUSH_MONITOR_TYPE
        Flag to indicate which push monitor controller will pick up for an upcoming push
        See Also:
        Constant Field Values
      • PARTICIPANT_MESSAGE_STORE_ENABLED

        public static final java.lang.String PARTICIPANT_MESSAGE_STORE_ENABLED
        Flag to enable the participant message store setup and write operations to the store.
        See Also:
        Constant Field Values
      • CONTROLLER_SYSTEM_SCHEMA_CLUSTER_NAME

        public static final java.lang.String CONTROLLER_SYSTEM_SCHEMA_CLUSTER_NAME
        The name of the cluster that should host the special stores used to serve system schemas.
        See Also:
        Constant Field Values
      • ADMIN_HELIX_MESSAGING_CHANNEL_ENABLED

        public static final java.lang.String ADMIN_HELIX_MESSAGING_CHANNEL_ENABLED
        Flag to enable the controller to send kill push job helix messages to the storage node upon consuming kill push job admin messages.
        See Also:
        Constant Field Values
      • PARTICIPANT_MESSAGE_CONSUMPTION_DELAY_MS

        public static final java.lang.String PARTICIPANT_MESSAGE_CONSUMPTION_DELAY_MS
        Minimum delay between each cycle where the storage node polls the participant message store to see if any of its ongoing push job has been killed.
        See Also:
        Constant Field Values
      • ROUTER_STATEFUL_HEALTHCHECK_ENABLED

        public static final java.lang.String ROUTER_STATEFUL_HEALTHCHECK_ENABLED
        See Also:
        Constant Field Values
      • ROUTER_UNHEALTHY_PENDING_CONNECTION_THRESHOLD_PER_ROUTE

        public static final java.lang.String ROUTER_UNHEALTHY_PENDING_CONNECTION_THRESHOLD_PER_ROUTE
        Maximum number of pending router request per storage node after which router concludes that host to be unhealthy and stops sending further request to it..
        See Also:
        Constant Field Values
      • ROUTER_PENDING_CONNECTION_RESUME_THRESHOLD_PER_ROUTE

        public static final java.lang.String ROUTER_PENDING_CONNECTION_RESUME_THRESHOLD_PER_ROUTE
        This is the threshold for pending request queue depth per storage node after which router resumes sending requests once a storage node which was previously marked unhealthy due to high ROUTER_UNHEALTHY_PENDING_CONNECTION_THRESHOLD_PER_ROUTE
        See Also:
        Constant Field Values
      • ROUTER_PER_NODE_CLIENT_ENABLED

        public static final java.lang.String ROUTER_PER_NODE_CLIENT_ENABLED
        Enables HttpAsyncClient allocation per storage node.
        See Also:
        Constant Field Values
      • ROUTER_PER_NODE_CLIENT_THREAD_COUNT

        public static final java.lang.String ROUTER_PER_NODE_CLIENT_THREAD_COUNT
        See Also:
        Constant Field Values
      • ROUTER_HTTP2_INBOUND_ENABLED

        public static final java.lang.String ROUTER_HTTP2_INBOUND_ENABLED
        Whether to support http/2 inbound request. When this feature is enabled, the Router will support both http/1.1 and http/2.
        See Also:
        Constant Field Values
      • ROUTER_HTTP2_MAX_CONCURRENT_STREAMS

        public static final java.lang.String ROUTER_HTTP2_MAX_CONCURRENT_STREAMS
        Indicates the maximum number of concurrent streams that the sender will allow. This limit is directional: it applies to the number of streams that the sender permits the receiver to create.
        See Also:
        Constant Field Values
      • ROUTER_HTTP2_MAX_FRAME_SIZE

        public static final java.lang.String ROUTER_HTTP2_MAX_FRAME_SIZE
        Indicates the size of the largest frame payload that the sender is willing to receive, in octets.
        See Also:
        Constant Field Values
      • ROUTER_HTTP2_INITIAL_WINDOW_SIZE

        public static final java.lang.String ROUTER_HTTP2_INITIAL_WINDOW_SIZE
        Indicates the sender's initial window size (in octets) for stream-level flow control.
        See Also:
        Constant Field Values
      • ROUTER_HTTP2_HEADER_TABLE_SIZE

        public static final java.lang.String ROUTER_HTTP2_HEADER_TABLE_SIZE
        Allows the sender to inform the remote endpoint of the maximum size of the header compression table used to decode header blocks, in octets.
        See Also:
        Constant Field Values
      • ROUTER_HTTP2_MAX_HEADER_LIST_SIZE

        public static final java.lang.String ROUTER_HTTP2_MAX_HEADER_LIST_SIZE
        This advisory setting informs a peer of the maximum size of header list that the sender is prepared to accept, in octets. The value is based on the uncompressed size of header fields, including the length of the name and value in octets plus an overhead of 32 octets for each header field.
        See Also:
        Constant Field Values
      • ROUTER_HTTP_CLIENT_OPENSSL_ENABLED

        public static final java.lang.String ROUTER_HTTP_CLIENT_OPENSSL_ENABLED
        Whether to enable openssl in the Router http client when talking to server.
        See Also:
        Constant Field Values
      • SERVER_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS

        public static final java.lang.String SERVER_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS
        In Leader/Follower state transition model, in order to avoid split brain problem (multiple leaders) as much as possible, the newly promoted leader should keep checking whether there is any new messages from the old leader in the version topic, and wait for some time (5 minutes by default) after the last message consumed before switching to leader role and potential starts producing to the version topic. Basically, the wait time could help us avoid the scenario that more than one replica is producing to the version topic.
        See Also:
        Constant Field Values
      • SERVER_SYSTEM_STORE_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS

        public static final java.lang.String SERVER_SYSTEM_STORE_PROMOTION_TO_LEADER_REPLICA_DELAY_SECONDS
        The system store, such as replica status related requires fast leadership fail over to avoid the stable info in system store, which could affect the request routing in the read path. Since we do have a way to correct the unordered data if it really happens, such as produce a full snapshot periodically, but the freshness is very important.
        See Also:
        Constant Field Values
      • SERVER_HTTP2_INBOUND_ENABLED

        public static final java.lang.String SERVER_HTTP2_INBOUND_ENABLED
        Whether to support http/2 inbound request. When this feature is enabled, the server will support both http/1.1 and http/2.
        See Also:
        Constant Field Values
      • SERVER_HTTP2_MAX_CONCURRENT_STREAMS

        public static final java.lang.String SERVER_HTTP2_MAX_CONCURRENT_STREAMS
        Indicates the maximum number of concurrent streams that the sender will allow. This limit is directional: it applies to the number of streams that the sender permits the receiver to create.
        See Also:
        Constant Field Values
      • SERVER_HTTP2_MAX_FRAME_SIZE

        public static final java.lang.String SERVER_HTTP2_MAX_FRAME_SIZE
        Indicates the size of the largest frame payload that the sender is willing to receive, in octets.
        See Also:
        Constant Field Values
      • SERVER_HTTP2_INITIAL_WINDOW_SIZE

        public static final java.lang.String SERVER_HTTP2_INITIAL_WINDOW_SIZE
        Indicates the sender's initial window size (in octets) for stream-level flow control.
        See Also:
        Constant Field Values
      • SERVER_HTTP2_HEADER_TABLE_SIZE

        public static final java.lang.String SERVER_HTTP2_HEADER_TABLE_SIZE
        Allows the sender to inform the remote endpoint of the maximum size of the header compression table used to decode header blocks, in octets.
        See Also:
        Constant Field Values
      • SERVER_HTTP2_MAX_HEADER_LIST_SIZE

        public static final java.lang.String SERVER_HTTP2_MAX_HEADER_LIST_SIZE
        This advisory setting informs a peer of the maximum size of header list that the sender is prepared to accept, in octets. The value is based on the uncompressed size of header fields, including the length of the name and value in octets plus an overhead of 32 octets for each header field.
        See Also:
        Constant Field Values
      • CONTROLLER_SSL_ENABLED

        public static final java.lang.String CONTROLLER_SSL_ENABLED
        This config defines whether SSL is enabled in controller.
        See Also:
        Constant Field Values
      • CONTROLLER_CLUSTER_LEADER_HAAS

        public static final java.lang.String CONTROLLER_CLUSTER_LEADER_HAAS
        Flag to indicate if the controller cluster leader will be amongst one of the local Helix as a library controllers or a Helix as a service controller running remotely.
        See Also:
        Constant Field Values
      • CONTROLLER_HAAS_SUPER_CLUSTER_NAME

        public static final java.lang.String CONTROLLER_HAAS_SUPER_CLUSTER_NAME
        The super cluster name for HAAS. This config is required if HAAS is enabled for the creation of helix clusters.
        See Also:
        Constant Field Values
      • CONTROLLER_ENABLE_BATCH_PUSH_FROM_ADMIN_IN_CHILD

        public static final java.lang.String CONTROLLER_ENABLE_BATCH_PUSH_FROM_ADMIN_IN_CHILD
        Whether to enable batch push (including GF job) from Admin in Child Controller. In theory, we should disable batch push in Child Controller no matter what, but the fact is that today there are many tests, which are doing batch pushes to an individual cluster setup (only Child Controller), so disabling batch push from Admin in Child Controller will require a lot of refactoring. So the current strategy is to enable it by default, but disable it in EI and PROD.
        See Also:
        Constant Field Values
      • KEY_VALUE_PROFILING_ENABLED

        public static final java.lang.String KEY_VALUE_PROFILING_ENABLED
        A config that turns the key/value profiling stats on and off. This config can be placed in both Router and SNs and it is off by default. When switching it on, We will emit a fine grained histogram that reflects the distribution of key and value size. Since this will be run in the critical read path and it will emit additional ~20 stats, please be cautious when turning it on.
        See Also:
        Constant Field Values
      • VENICE_STORAGE_CLUSTER_LEADER_HAAS

        public static final java.lang.String VENICE_STORAGE_CLUSTER_LEADER_HAAS
        See Also:
        Constant Field Values
      • PARTITIONER_CLASS

        public static final java.lang.String PARTITIONER_CLASS
        A config specifies which partitioning scheme should be used by VenicePushJob.
        See Also:
        Constant Field Values
      • AMPLIFICATION_FACTOR

        public static final java.lang.String AMPLIFICATION_FACTOR
        A configs of over-partitioning factor number of Kafka partitions in each partition
        See Also:
        Constant Field Values
      • INSTANCE_ID

        public static final java.lang.String INSTANCE_ID
        A unique id that can represent this instance
        See Also:
        Constant Field Values
      • ROUTER_DICTIONARY_RETRIEVAL_TIME_MS

        public static final java.lang.String ROUTER_DICTIONARY_RETRIEVAL_TIME_MS
        Maximum time allowed for router to download dictionary from Storage nodes.
        See Also:
        Constant Field Values
      • ROUTER_DICTIONARY_PROCESSING_THREADS

        public static final java.lang.String ROUTER_DICTIONARY_PROCESSING_THREADS
        Number of threads that the Router will use to wait for dictionary to download from storage nodes and process it.
        See Also:
        Constant Field Values
      • KAFKA_WRITE_ONLY_ADMIN_CLASS

        public static final java.lang.String KAFKA_WRITE_ONLY_ADMIN_CLASS
        Fully-qualified class name to use for Kafka write-only admin operations.
        See Also:
        Constant Field Values
      • KAFKA_READ_ONLY_ADMIN_CLASS

        public static final java.lang.String KAFKA_READ_ONLY_ADMIN_CLASS
        Fully-qualified class name to use for Kafka read-only admin operations.
        See Also:
        Constant Field Values
      • HELIX_HYBRID_STORE_QUOTA_ENABLED

        public static final java.lang.String HELIX_HYBRID_STORE_QUOTA_ENABLED
        A config that determines whether to use Helix customized view for hybrid store quota
        See Also:
        Constant Field Values
      • SERVER_SHUTDOWN_DISK_UNHEALTHY_TIME_MS

        public static final java.lang.String SERVER_SHUTDOWN_DISK_UNHEALTHY_TIME_MS
        A time after which a bad SSD will trigger server shutdown.
        See Also:
        Constant Field Values
      • ROUTER_EARLY_THROTTLE_ENABLED

        public static final java.lang.String ROUTER_EARLY_THROTTLE_ENABLED
        Turns on early router throttling before allocating most of the router resources.
        See Also:
        Constant Field Values
      • ROUTER_HEART_BEAT_ENABLED

        public static final java.lang.String ROUTER_HEART_BEAT_ENABLED
        Disable router heart-beat job which marks hosts as unhealthy.
        See Also:
        Constant Field Values
      • ROUTER_HTTP_CLIENT5_POOL_SIZE

        public static final java.lang.String ROUTER_HTTP_CLIENT5_POOL_SIZE
        HttpClient5 pool size.
        See Also:
        Constant Field Values
      • ROUTER_HTTP_CLIENT5_TOTAL_IO_THREAD_COUNT

        public static final java.lang.String ROUTER_HTTP_CLIENT5_TOTAL_IO_THREAD_COUNT
        Total IO thread count for HttpClient5 pool.
        See Also:
        Constant Field Values
      • ROUTER_HTTP_CLIENT5_SKIP_CIPHER_CHECK_ENABLED

        public static final java.lang.String ROUTER_HTTP_CLIENT5_SKIP_CIPHER_CHECK_ENABLED
        Whether to skip the cipher check when using Httpclient5.
        See Also:
        Constant Field Values
      • ROUTER_HTTPASYNCCLIENT_CLIENT_POOL_THREAD_COUNT

        public static final java.lang.String ROUTER_HTTPASYNCCLIENT_CLIENT_POOL_THREAD_COUNT
        Number of IO threads used for AHAC client.
        See Also:
        Constant Field Values
      • ERROR_PARTITION_AUTO_RESET_LIMIT

        public static final java.lang.String ERROR_PARTITION_AUTO_RESET_LIMIT
        Maximum number of times controller will automatically reset an error partition for the current/serving version to mitigate impact of transient or long running issues during re-balance or restart.
        See Also:
        Constant Field Values
      • ERROR_PARTITION_PROCESSING_CYCLE_DELAY

        public static final java.lang.String ERROR_PARTITION_PROCESSING_CYCLE_DELAY
        The delay between each cycle where we iterate over all applicable resources and their partition to reset error partitions and collect data on the effectiveness of previous resets.
        See Also:
        Constant Field Values
      • TERMINAL_STATE_TOPIC_CHECK_DELAY_MS

        public static final java.lang.String TERMINAL_STATE_TOPIC_CHECK_DELAY_MS
        Delay between each cycle where the checker will iterate over existing topics that are yet to be truncated and poll their job status until they reach terminal state to ensure version topics in parent fabric are truncated in a timely manner.
        See Also:
        Constant Field Values
      • CLIENT_USE_SYSTEM_STORE_REPOSITORY

        public static final java.lang.String CLIENT_USE_SYSTEM_STORE_REPOSITORY
        A config for Da-Vinci clients to use system store based repositories or Zk based repositories.
        See Also:
        Constant Field Values
      • CLIENT_SYSTEM_STORE_REPOSITORY_REFRESH_INTERVAL_SECONDS

        public static final java.lang.String CLIENT_SYSTEM_STORE_REPOSITORY_REFRESH_INTERVAL_SECONDS
        The refresh interval for system store repositories that rely on periodic polling.
        See Also:
        Constant Field Values
      • CONTROLLER_DISABLE_PARENT_TOPIC_TRUNCATION_UPON_COMPLETION

        public static final java.lang.String CONTROLLER_DISABLE_PARENT_TOPIC_TRUNCATION_UPON_COMPLETION
        Test only config used to disable parent topic truncation upon job completion. This is needed because kafka cluster in test environment is shared between parent and child controllers. Truncating topic upon completion will confuse child controllers in certain scenarios.
        See Also:
        Constant Field Values
      • D2_ZK_HOSTS_ADDRESS

        public static final java.lang.String D2_ZK_HOSTS_ADDRESS
        ZooKeeper address of d2 client.
        See Also:
        Constant Field Values
      • PUSH_STATUS_STORE_ENABLED

        public static final java.lang.String PUSH_STATUS_STORE_ENABLED
        Config to control if push status store is enabled and should be initialized
        See Also:
        Constant Field Values
      • DAVINCI_PUSH_STATUS_SCAN_ENABLED

        public static final java.lang.String DAVINCI_PUSH_STATUS_SCAN_ENABLED
        See Also:
        Constant Field Values
      • DAVINCI_PUSH_STATUS_SCAN_INTERVAL_IN_SECONDS

        public static final java.lang.String DAVINCI_PUSH_STATUS_SCAN_INTERVAL_IN_SECONDS
        See Also:
        Constant Field Values
      • DAVINCI_PUSH_STATUS_SCAN_THREAD_NUMBER

        public static final java.lang.String DAVINCI_PUSH_STATUS_SCAN_THREAD_NUMBER
        See Also:
        Constant Field Values
      • DAVINCI_PUSH_STATUS_SCAN_NO_REPORT_RETRY_MAX_ATTEMPTS

        public static final java.lang.String DAVINCI_PUSH_STATUS_SCAN_NO_REPORT_RETRY_MAX_ATTEMPTS
        Max retry when not receiving any DaVinci status report. This is mainly for testing purpose since in local integration test, the push job runs too fast in the backend, and no DaVinci status report will mark the push job succeed right away.
        See Also:
        Constant Field Values
      • DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_COUNT

        public static final java.lang.String DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_COUNT
        See Also:
        Constant Field Values
      • DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_RATIO

        public static final java.lang.String DAVINCI_PUSH_STATUS_SCAN_MAX_OFFLINE_INSTANCE_RATIO
        See Also:
        Constant Field Values
      • CONTROLLER_ZK_SHARED_DAVINCI_PUSH_STATUS_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED

        public static final java.lang.String CONTROLLER_ZK_SHARED_DAVINCI_PUSH_STATUS_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED
        See Also:
        Constant Field Values
      • PUSH_STATUS_STORE_HEARTBEAT_INTERVAL_IN_SECONDS

        public static final java.lang.String PUSH_STATUS_STORE_HEARTBEAT_INTERVAL_IN_SECONDS
        Interval for Da Vinci clients to send heartbeats.
        See Also:
        Constant Field Values
      • PUSH_STATUS_INSTANCE_NAME_SUFFIX

        public static final java.lang.String PUSH_STATUS_INSTANCE_NAME_SUFFIX
        Config to control what's the suffix for Da Vinci instance which is reporting push status and heartbeats. By default, it is process PID if not specified, but note that PID is subject to change upon instance restart.
        See Also:
        Constant Field Values
      • PUSH_STATUS_STORE_HEARTBEAT_EXPIRATION_TIME_IN_SECONDS

        public static final java.lang.String PUSH_STATUS_STORE_HEARTBEAT_EXPIRATION_TIME_IN_SECONDS
        The expiration timeout. If an instance not sending heartbeats for over the expiration time, it will be considered as stale.
        See Also:
        Constant Field Values
      • USE_DA_VINCI_SPECIFIC_EXECUTION_STATUS_FOR_ERROR

        public static final java.lang.String USE_DA_VINCI_SPECIFIC_EXECUTION_STATUS_FOR_ERROR
        when enabled, Da Vinci Clients returns specific status codes to indicate the type of ingestion failure rather than a generic com.linkedin.venice.pushmonitor.ExecutionStatus.ERROR
        See Also:
        Constant Field Values
      • ROUTER_CLIENT_SSL_HANDSHAKE_THREADS

        public static final java.lang.String ROUTER_CLIENT_SSL_HANDSHAKE_THREADS
        The number of threads that will be used to perform SSL handshakes between clients and a router.
        See Also:
        Constant Field Values
      • ROUTER_CLIENT_SSL_HANDSHAKE_QUEUE_CAPACITY

        public static final java.lang.String ROUTER_CLIENT_SSL_HANDSHAKE_QUEUE_CAPACITY
        Config to control the queue capacity for the thread pool executor used for ssl handshake between clients and a router.
        See Also:
        Constant Field Values
      • ROUTER_READ_QUOTA_THROTTLING_LEASE_TIMEOUT_MS

        public static final java.lang.String ROUTER_READ_QUOTA_THROTTLING_LEASE_TIMEOUT_MS
        Lease timeout for leaving quota disabled for a router. If quota was disabled through an API, it will be reset after lease expiry.
        See Also:
        Constant Field Values
      • CONTROLLER_SYSTEM_STORE_ACL_SYNCHRONIZATION_DELAY_MS

        public static final java.lang.String CONTROLLER_SYSTEM_STORE_ACL_SYNCHRONIZATION_DELAY_MS
        The delay in ms between each synchronization attempt between the Venice store acls and its corresponding system store acls if any synchronization is needed.
        See Also:
        Constant Field Values
      • LEAKED_PUSH_STATUS_CLEAN_UP_SERVICE_SLEEP_INTERVAL_MS

        public static final java.lang.String LEAKED_PUSH_STATUS_CLEAN_UP_SERVICE_SLEEP_INTERVAL_MS
        This config defines the sleep interval in leaked push status clean up service.
        See Also:
        Constant Field Values
      • LEAKED_RESOURCE_ALLOWED_LINGER_TIME_MS

        public static final java.lang.String LEAKED_RESOURCE_ALLOWED_LINGER_TIME_MS
        This config defines the allowed linger time for a leaked resource.
        See Also:
        Constant Field Values
      • CLIENT_USE_DA_VINCI_BASED_SYSTEM_STORE_REPOSITORY

        public static final java.lang.String CLIENT_USE_DA_VINCI_BASED_SYSTEM_STORE_REPOSITORY
        This config controls whether to use da-vinci based implementation of the system store repository when CLIENT_USE_SYSTEM_STORE_REPOSITORY is set to true. By default the thin-client based implementation will be used.
        See Also:
        Constant Field Values
      • CONTROLLER_DISABLE_PARENT_REQUEST_TOPIC_FOR_STREAM_PUSHES

        public static final java.lang.String CONTROLLER_DISABLE_PARENT_REQUEST_TOPIC_FOR_STREAM_PUSHES
        See Also:
        Constant Field Values
      • CONTROLLER_DEFAULT_READ_QUOTA_PER_ROUTER

        public static final java.lang.String CONTROLLER_DEFAULT_READ_QUOTA_PER_ROUTER
        See Also:
        Constant Field Values
      • LOCAL_REGION_NAME

        public static final java.lang.String LOCAL_REGION_NAME
        This config will specify the region name of a controller; the region name can be customized by Venice internal.
        See Also:
        Constant Field Values
      • CONTROLLER_AUTO_MATERIALIZE_META_SYSTEM_STORE

        public static final java.lang.String CONTROLLER_AUTO_MATERIALIZE_META_SYSTEM_STORE
        This config controls whether to make an empty push to materialize meta system store for newly created user stores if possible.
        See Also:
        Constant Field Values
      • CONTROLLER_AUTO_MATERIALIZE_DAVINCI_PUSH_STATUS_SYSTEM_STORE

        public static final java.lang.String CONTROLLER_AUTO_MATERIALIZE_DAVINCI_PUSH_STATUS_SYSTEM_STORE
        See Also:
        Constant Field Values
      • REPLICATION_METADATA_VERSION

        public static final java.lang.String REPLICATION_METADATA_VERSION
        This will indicate which ReplicationMetadataSchemaGenerator version to use to generate replication metadata schema. This config should be set on a per-cluster level, meaning that each cluster can have its own RMD version ID.
        See Also:
        Constant Field Values
      • OFFSET_LAG_DELTA_RELAX_FACTOR_FOR_FAST_ONLINE_TRANSITION_IN_RESTART

        public static final java.lang.String OFFSET_LAG_DELTA_RELAX_FACTOR_FOR_FAST_ONLINE_TRANSITION_IN_RESTART
        See Also:
        Constant Field Values
      • SERVER_KAFKA_CONSUMER_OFFSET_COLLECTION_ENABLED

        public static final java.lang.String SERVER_KAFKA_CONSUMER_OFFSET_COLLECTION_ENABLED
        Enable offset collection for kafka topic partition from kafka consumer metrics.
        See Also:
        Constant Field Values
      • SERVER_SCHEMA_PRESENCE_CHECK_ENABLED

        public static final java.lang.String SERVER_SCHEMA_PRESENCE_CHECK_ENABLED
        This indicates if server will perform the schema presence check or not. By default it is set to true.
        See Also:
        Constant Field Values
      • CONTROLLER_JETTY_CONFIG_OVERRIDE_PREFIX

        public static final java.lang.String CONTROLLER_JETTY_CONFIG_OVERRIDE_PREFIX
        Prefix of configs to configure Jetty server in Controller.
        See Also:
        Constant Field Values
      • ROUTER_META_STORE_SHADOW_READ_ENABLED

        public static final java.lang.String ROUTER_META_STORE_SHADOW_READ_ENABLED
        The number of records
        See Also:
        Constant Field Values
      • KAFKA_CLUSTER_MAP_KEY_NAME

        public static final java.lang.String KAFKA_CLUSTER_MAP_KEY_NAME
        Defines the key names in venice.server.kafkaClustersMap
        See Also:
        Constant Field Values
      • KAFKA_CLUSTER_MAP_KEY_URL

        public static final java.lang.String KAFKA_CLUSTER_MAP_KEY_URL
        See Also:
        Constant Field Values
      • KAFKA_CLUSTER_MAP_KEY_OTHER_URLS

        public static final java.lang.String KAFKA_CLUSTER_MAP_KEY_OTHER_URLS
        See Also:
        Constant Field Values
      • KAFKA_CLUSTER_MAP_SECURITY_PROTOCOL

        public static final java.lang.String KAFKA_CLUSTER_MAP_SECURITY_PROTOCOL
        See Also:
        Constant Field Values
      • SERVER_ENABLE_LIVE_CONFIG_BASED_KAFKA_THROTTLING

        public static final java.lang.String SERVER_ENABLE_LIVE_CONFIG_BASED_KAFKA_THROTTLING
        See Also:
        Constant Field Values
      • CONCURRENT_INIT_ROUTINES_ENABLED

        public static final java.lang.String CONCURRENT_INIT_ROUTINES_ENABLED
        Enable the concurrent execution of the controllers' init routines, which are executed when a controller becomes a cluster leader.
        See Also:
        Constant Field Values
      • SERVER_INGESTION_CHECKPOINT_DURING_GRACEFUL_SHUTDOWN_ENABLED

        public static final java.lang.String SERVER_INGESTION_CHECKPOINT_DURING_GRACEFUL_SHUTDOWN_ENABLED
        A config to control graceful shutdown. True: servers will flush all remain data in producers buffers and drainer queues, and persist all data including offset metadata and producer states into disk False: servers will not flush any data during shutdown. After restart, servers will resume ingestion from the last checkpoint.
        See Also:
        Constant Field Values
      • USE_PUSH_STATUS_STORE_FOR_INCREMENTAL_PUSH

        public static final java.lang.String USE_PUSH_STATUS_STORE_FOR_INCREMENTAL_PUSH
        A config to control which status store to use for fetching incremental push job status from the controller. This config should be removed once the migration of push status to push status system store is complete. True: use push system status store False: use zookeeper store
        See Also:
        Constant Field Values
      • META_STORE_WRITER_CLOSE_TIMEOUT_MS

        public static final java.lang.String META_STORE_WRITER_CLOSE_TIMEOUT_MS
        A config to control the maximum time spent on closing the meta store writer.
        See Also:
        Constant Field Values
      • META_STORE_WRITER_CLOSE_CONCURRENCY

        public static final java.lang.String META_STORE_WRITER_CLOSE_CONCURRENCY
        A config to control the maximum number of concurrent meta store writer close operations.
        See Also:
        Constant Field Values
      • SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_ENABLED

        public static final java.lang.String SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_ENABLED
        A config to control whether VeniceServer will optimize the database for the backup version to free up memory resources occupied. TODO: explore to apply this feature to DVC as well. This feature should be very useful for RocksDB plaintable to unload the mmapped memory and it will be useful for RocksDB block-based format as well to evict the unused index/filters from the shared block cache.
        See Also:
        Constant Field Values
      • SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_NO_READ_THRESHOLD_SECONDS

        public static final java.lang.String SERVER_OPTIMIZE_DATABASE_FOR_BACKUP_VERSION_NO_READ_THRESHOLD_SECONDS
        A config to control the no read threshold when the database optimization should kick in.
        See Also:
        Constant Field Values
      • SERVER_OPTIMIZE_DATABASE_SERVICE_SCHEDULE_INTERNAL_SECONDS

        public static final java.lang.String SERVER_OPTIMIZE_DATABASE_SERVICE_SCHEDULE_INTERNAL_SECONDS
        Schedule interval for database optimization service.
        See Also:
        Constant Field Values
      • UNREGISTER_METRIC_FOR_DELETED_STORE_ENABLED

        public static final java.lang.String UNREGISTER_METRIC_FOR_DELETED_STORE_ENABLED
        A config that determines whether to unregister per store metrics when a store is deleted. Default is false.
        See Also:
        Constant Field Values
      • FORCE_LEADER_ERROR_REPLICA_FAIL_OVER_ENABLED

        public static final java.lang.String FORCE_LEADER_ERROR_REPLICA_FAIL_OVER_ENABLED
        Config to enable single leader replica disabling.
        See Also:
        Constant Field Values
      • IDENTITY_PARSER_CLASS

        public static final java.lang.String IDENTITY_PARSER_CLASS
        A config to specify the class to use to parse identities at runtime
        See Also:
        Constant Field Values
      • VENICE_PARTITIONERS

        public static final java.lang.String VENICE_PARTITIONERS
        Specifies a list of partitioners venice supported. It contains a string of concatenated partitioner class names separated by comma.
        See Also:
        Constant Field Values
      • VALIDATE_VENICE_INTERNAL_SCHEMA_VERSION

        public static final java.lang.String VALIDATE_VENICE_INTERNAL_SCHEMA_VERSION
        Config to check whether the protocol versions used at runtime are valid in Venice backend; if not, fail fast. Used by Samza jobs and Da Vinci clients. Default value should be true. Turn off the config in where access to routers is not feasible.
        See Also:
        Constant Field Values
      • FAST_AVRO_FIELD_LIMIT_PER_METHOD

        public static final java.lang.String FAST_AVRO_FIELD_LIMIT_PER_METHOD
        Config to control the maximum number of fields per method in a fast-avro generated deserializer. Can be useful if the JIT limit of 8 KB of bytecode is reached. An alternative is to use the -XX:-DontCompileHugeMethods JVM flag but that can have other side effects, so it may not be preferred.
        See Also:
        Constant Field Values
      • SERVER_SSL_HANDSHAKE_THREAD_POOL_SIZE

        public static final java.lang.String SERVER_SSL_HANDSHAKE_THREAD_POOL_SIZE
        Config to control the number of threads in the thread pool executor used for ssl handshake in servers. The purpose is to limit the concurrency of ssl handshakes. The feature to use a thread pool executor for handling ssl handshakes is disabled if the value of this config is <= 0. The default value is 0.
        See Also:
        Constant Field Values
      • SERVER_SSL_HANDSHAKE_QUEUE_CAPACITY

        public static final java.lang.String SERVER_SSL_HANDSHAKE_QUEUE_CAPACITY
        Config to control the queue capacity for the thread pool executor used for ssl handshake in servers.
        See Also:
        Constant Field Values
      • CLIENT_PRODUCER_THREAD_NUM

        public static final java.lang.String CLIENT_PRODUCER_THREAD_NUM
        Number of threads for online Venice producer controlling the number of concurrent write operations.
        See Also:
        Constant Field Values
      • CLIENT_PRODUCER_SCHEMA_REFRESH_INTERVAL_SECONDS

        public static final java.lang.String CLIENT_PRODUCER_SCHEMA_REFRESH_INTERVAL_SECONDS
        The refresh interval for online producer to refresh value schemas and update schemas that rely on periodic polling.
        See Also:
        Constant Field Values
      • INGESTION_MEMORY_LIMIT

        public static final java.lang.String INGESTION_MEMORY_LIMIT
        See Also:
        Constant Field Values
      • INGESTION_MLOCK_ENABLED

        public static final java.lang.String INGESTION_MLOCK_ENABLED
        Whether the ingestion is using mlock or not. Currently, this option is only meaningful for DaVinci use cases. Actually, this config option is being actively used, and it is a placeholder for the future optimization. The memory limit logic implemented today is assuming mlock usage, and to make it backward compatible when we want to do more optimization for non-mlock usage, we will ask the mlock user to enable this flag.
        See Also:
        Constant Field Values
      • INGESTION_MEMORY_LIMIT_STORE_LIST

        public static final java.lang.String INGESTION_MEMORY_LIMIT_STORE_LIST
        Only applies the memory limiter to the stores listed in this config. This is mainly used for testing purpose since ultimately, we want to enforce memory limiter against all the stores to avoid node crash. Empty config means ingestion memory limiter will apply to all the stores.
        See Also:
        Constant Field Values
      • DIV_PRODUCER_STATE_MAX_AGE_MS

        public static final java.lang.String DIV_PRODUCER_STATE_MAX_AGE_MS
        The maximum age (in milliseconds) of producer state retained by Data Ingestion Validation. Tuning this can prevent OOMing in cases where there is a lot of historical churn in RT producers. The age of a given producer's state is defined as: most_recent_timestamp_of_all_producers - most_recent_timestamp_of_given_producer This math is computed within a single partition, not across partitions. If enabled, the clearing of old state will happen when subscribing to a partition (e.g. on server start up), and prior to syncing progress to disk (e.g. when SERVER_DATABASE_SYNC_BYTES_INTERNAL_FOR_DEFERRED_WRITE_MODE is reached). Old state clearing is disabled if this config is set to -1.
        See Also:
        Constant Field Values
      • PUB_SUB_ADMIN_ADAPTER_FACTORY_CLASS

        public static final java.lang.String PUB_SUB_ADMIN_ADAPTER_FACTORY_CLASS
        See Also:
        Constant Field Values
      • PUB_SUB_PRODUCER_ADAPTER_FACTORY_CLASS

        public static final java.lang.String PUB_SUB_PRODUCER_ADAPTER_FACTORY_CLASS
        See Also:
        Constant Field Values
      • PUB_SUB_CONSUMER_ADAPTER_FACTORY_CLASS

        public static final java.lang.String PUB_SUB_CONSUMER_ADAPTER_FACTORY_CLASS
        See Also:
        Constant Field Values
      • ROUTER_PRINCIPAL_NAME

        public static final java.lang.String ROUTER_PRINCIPAL_NAME
        Venice router's principal name used for ssl. Default should contain "venice-router".
        See Also:
        Constant Field Values
      • SERVER_INGESTION_HEARTBEAT_INTERVAL_MS

        public static final java.lang.String SERVER_INGESTION_HEARTBEAT_INTERVAL_MS
        The time interval in milliseconds for leader replica to send heartbeat to RT topic for consumers (including the leader) to keep its latest processed upstream RT offset up-to-date in case when the RT topic ends with SOS, EOS or skipped records.
        See Also:
        Constant Field Values
      • SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_ENABLED

        public static final java.lang.String SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_ENABLED
        Whether to check LeaderCompleteState in the follower replica and davinci replica before marking the follower completed. This is to avoid the case that the follower replica is marked completed before the leader replica and transitions to leader if the leader replicas goes down.

        Default to false. Should be enabled only after Venice tag 0.4.154 is fully rolled out.

        See Also:
        Constant Field Values
      • SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_VALID_INTERVAL_MS

        public static final java.lang.String SERVER_LEADER_COMPLETE_STATE_CHECK_IN_FOLLOWER_VALID_INTERVAL_MS
        Follower replicas and DavinciClient will only consider heartbeats received within this time window to mark themselves as completed. This is to avoid the cases that the follower replica is marked completed based on the old heartbeat messages from a previous leader replica. Note that the leader replica keeps sending the leader completed headers in every heartbeat messages which allows the follower replica the liberty to decide based on the freshness of the heartbeat messages to avoid stale data from some edge cases scenarios.
        See Also:
        Constant Field Values
      • SERVER_STUCK_CONSUMER_REPAIR_ENABLED

        public static final java.lang.String SERVER_STUCK_CONSUMER_REPAIR_ENABLED
        Whether to enable stuck consumer repair in Server.
        See Also:
        Constant Field Values
      • SERVER_STUCK_CONSUMER_REPAIR_INTERVAL_SECOND

        public static final java.lang.String SERVER_STUCK_CONSUMER_REPAIR_INTERVAL_SECOND
        Server stuck consumer detection interval.
        See Also:
        Constant Field Values
      • SERVER_STUCK_CONSUMER_REPAIR_THRESHOLD_SECOND

        public static final java.lang.String SERVER_STUCK_CONSUMER_REPAIR_THRESHOLD_SECOND
        Server stuck consumer repair threshold.
        See Also:
        Constant Field Values
      • SERVER_NON_EXISTING_TOPIC_INGESTION_TASK_KILL_THRESHOLD_SECOND

        public static final java.lang.String SERVER_NON_EXISTING_TOPIC_INGESTION_TASK_KILL_THRESHOLD_SECOND
        When to kill the ingestion task if the topic doesn't exist for the configured period of time.
        See Also:
        Constant Field Values
      • SERVER_DEDICATED_CONSUMER_POOL_FOR_AA_WC_LEADER_ENABLED

        public static final java.lang.String SERVER_DEDICATED_CONSUMER_POOL_FOR_AA_WC_LEADER_ENABLED
        Handling AA or WC stores is expensive because of RocksDB lookup, and this following feature will handle these writes in dedicated consumer pool, so that the full updates won't be affected by the heavy writes to these AA/WC stores.
        See Also:
        Constant Field Values
      • SERVER_DEDICATED_CONSUMER_POOL_SIZE_FOR_AA_WC_LEADER

        public static final java.lang.String SERVER_DEDICATED_CONSUMER_POOL_SIZE_FOR_AA_WC_LEADER
        See Also:
        Constant Field Values
      • SERVER_RECORD_LEVEL_METRICS_WHEN_BOOTSTRAPPING_CURRENT_VERSION_ENABLED

        public static final java.lang.String SERVER_RECORD_LEVEL_METRICS_WHEN_BOOTSTRAPPING_CURRENT_VERSION_ENABLED
        Whether to enable record-level metrics when bootstrapping current version. This feature will be mainly used by DaVinci to speed up bootstrapping.
        See Also:
        Constant Field Values