Package com.linkedin.davinci.config

Class VeniceStoreVersionConfig

java.lang.Object

com.linkedin.davinci.config.VeniceClusterConfig

com.linkedin.davinci.config.VeniceServerConfig

com.linkedin.davinci.config.VeniceStoreVersionConfig

public class VeniceStoreVersionConfig
extends VeniceServerConfig

class that maintains all properties that are not specific to a venice server and cluster. Includes individual store properties and other properties that can be overwritten.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.davinci.config.VeniceServerConfig

VeniceServerConfig.IncrementalPushStatusWriteMode

Field Summary

Fields inherited from class com.linkedin.davinci.config.VeniceServerConfig

MINIMUM_CONSUMER_NUM_IN_CONSUMER_POOL_PER_KAFKA_CLUSTER, readOnlyForBatchOnlyStoreEnabled

Constructor Summary

Constructors

Constructor	Description
VeniceStoreVersionConfig(String storeVersionName, VeniceProperties storeProperties)
VeniceStoreVersionConfig(String storeVersionName, VeniceProperties storeProperties, PersistenceType storePersistenceType)
VeniceStoreVersionConfig(String storeVersionName, VeniceProperties storeProperties, Map<String,Map<String,String>> kafkaClusterMap)

Method Summary

Modifier and Type	Method	Description
VeniceProperties	getPersistStorageEngineConfig()
PersistenceType	getStorePersistenceType()
String	getStoreVersionName()
boolean	isBlobTransferEnabled()
boolean	isRestoreDataPartitions()
boolean	isRestoreMetadataPartition()
boolean	isStorePersistenceTypeKnown()	For some store, the persistence type may not be known when constructing VeniceStoreVersionConfig, such as in `VeniceStateModelFactory#createNewStateModel`, when Helix wants to create a new state model for some store, it doesn't know the persistence type since it is possible that this store is an existing store, so the persistence type is decided by the on-disk type, and it is a new store, it will be decided by storage node config.
void	setBlobTransferEnabled(boolean blobTransferEnabled)
void	setRestoreDataPartitions(boolean restoreDataPartitions)
void	setRestoreMetadataPartition(boolean restoreMetadataPartition)
void	setStorePersistenceType(PersistenceType storePersistenceType)	This reason to create a `setter` for storePersistenceType since it is unknown sometimes when initializing VeniceStoreVersionConfig, such as in com.linkedin.venice.helix.VeniceStateModelFactory#createNewStateModel, The new state model could be created for a brand-new store or an existing store.

Methods inherited from class com.linkedin.davinci.config.VeniceServerConfig

enforceMemoryLimitInStore, extractIncPushStatusWriteMode, freezeIngestionIfReadyToServeOrLocalDataExists, getAaWCLeaderQuotaRecordsPerSecond, getAAWCWorkloadParallelProcessingThreadPoolSize, getAdaptiveThrottlerSignalIdleThreshold, getAdaptiveThrottlerSingleGetLatencyThreshold, getBatchReportEOIPEnabled, getBlobTransferDisabledOffsetLagThreshold, getBlobTransferMaxTimeoutInMin, getBlockingQueueType, getChannelOptionWriteBufferHighBytes, getComputeQueueCapacity, getConsumerPoolSizeForCurrentVersionAAWCLeader, getConsumerPoolSizeForCurrentVersionNonAAWCLeader, getConsumerPoolSizeForCurrentVersionSepRTLeader, getConsumerPoolSizeForNonCurrentVersionAAWCLeader, getConsumerPoolSizeForNonCurrentVersionNonAAWCLeader, getConsumerPoolSizePerKafkaCluster, getConsumerPoolStrategyType, getCurrentVersionAAWCLeaderQuotaRecordsPerSecond, getCurrentVersionNonAAWCLeaderQuotaRecordsPerSecond, getCurrentVersionSepRTLeaderQuotaRecordsPerSecond, getDatabaseLookupQueueCapacity, getDataBasePath, getDatabaseSyncBytesIntervalForDeferredWriteMode, getDatabaseSyncBytesIntervalForTransactionalMode, getDaVinciCurrentVersionBootstrappingQuotaBytesPerSecond, getDaVinciCurrentVersionBootstrappingQuotaRecordsPerSecond, getDaVinciPushStatusCheckIntervalInMs, getDedicatedConsumerPoolSizeForAAWCLeader, getDefaultMaxRecordSizeBytes, getDiskFullThreshold, getDiskHealthCheckIntervalInMS, getDiskHealthCheckTimeoutInMs, getDivProducerStateMaxAgeMs, getDrainerPoolSizeSortedInput, getDrainerPoolSizeUnsortedInput, getDvcP2pBlobTransferClientPort, getDvcP2pBlobTransferServerPort, getFastAvroFieldLimitPerMethod, getFastClassSchemaWarmupTimeout, getForkedProcessJvmArgList, getGrpcPort, getGrpcWorkerThreadCount, getHttp2HeaderTableSize, getHttp2InitialWindowSize, getHttp2MaxConcurrentStreams, getHttp2MaxFrameSize, getHttp2MaxHeaderListSize, getIdentityParserClassName, getIncrementalPushStatusWriteMode, getIngestionApplicationPort, getIngestionHeartbeatIntervalMs, getIngestionMemoryLimit, getIngestionMode, getIngestionServicePort, getIngestionTaskMaxIdleCount, getKafkaAdminClass, getKafkaConsumerConfigsForLocalConsumption, getKafkaConsumerConfigsForRemoteConsumption, getKafkaMaxPollRecords, getKafkaReadOnlyClass, getKafkaWriteOnlyClass, getLeaderCompleteStateCheckInFollowerValidIntervalMs, getLeaderFollowerThreadPoolStrategy, getLeakedResourceCleanUpIntervalInMS, getListenerHostname, getListenerPort, getLocalControllerD2ServiceName, getLocalControllerUrl, getLocalD2ZkHost, getMaxConcurrentSnapshotUser, getMaxFutureVersionLeaderFollowerStateTransitionThreadNumber, getMaxLeaderFollowerStateTransitionThreadNumber, getMaxRequestSize, getMaxWaitAfterUnsubscribeMs, getMetaStoreWriterCloseConcurrency, getMetaStoreWriterCloseTimeoutInMS, getNettyGracefulShutdownPeriodSeconds, getNettyIdleTimeInSeconds, getNettyWorkerThreadCount, getNodeCapacityInRcu, getNonCurrentVersionAAWCLeaderQuotaRecordsPerSecond, getNonCurrentVersionNonAAWCLeaderQuotaRecordsPerSecond, getNonExistingTopicCheckRetryIntervalSecond, getNonExistingTopicIngestionTaskKillThresholdSecond, getNumSchemaFastClassWarmup, getOffsetLagDeltaRelaxFactorForFastOnlineTransitionInRestart, getOptimizeDatabaseForBackupVersionNoReadThresholdMS, getOptimizeDatabaseServiceScheduleIntervalSeconds, getParallelBatchGetChunkSize, getParticipantMessageConsumptionDelayMs, getPartitionGracefulDropDelaySeconds, getPubSubClientsFactory, getPubSubConsumerPollRetryBackoffMs, getPubSubConsumerPollRetryTimes, getQuotaEnforcementCapacityMultiple, getQuotaEnforcementIntervalInMs, getRemoteIngestionRepairSleepInterval, getRestServiceStorageThreadNum, getRocksDBPath, getRocksDBServerConfig, getRouterConnectionWarmingDelayMs, getRouterPrincipalName, getServerComputeThreadNum, getServerMaxWaitForVersionInfo, getServerPromotionToLeaderReplicaDelayMs, getServerSystemStorePromotionToLeaderReplicaDelayMs, getSharedConsumerAssignmentStrategy, getSharedConsumerNonExistingTopicCleanupDelayMS, getSnapshotRetentionTimeInMin, getSsdHealthCheckShutdownTimeMs, getSslHandshakeQueueCapacity, getSslHandshakeThreadPoolSize, getStopConsumptionTimeoutInSeconds, getStorageNodeRateLimiterType, getStoreToEarlyTerminationThresholdMSMap, getStoreVersionQpsRateLimiterType, getStoreWriterBufferMemoryCapacity, getStoreWriterBufferNotifyDelta, getStoreWriterNumber, getStuckConsumerDetectionRepairThresholdSecond, getStuckConsumerRepairIntervalSecond, getSystemSchemaClusterName, getTopicManagerMetadataFetcherConsumerPoolSize, getTopicManagerMetadataFetcherThreadPoolSize, getTopicOffsetCheckIntervalMs, getZstdDictCompressionLevel, isAAWCWorkloadParallelProcessingEnabled, isAdaptiveThrottlerEnabled, isAutoCreateDataPath, isBlobTransferManagerEnabled, isComputeFastAvroEnabled, isDatabaseChecksumVerificationEnabled, isDatabaseMemoryStatsEnabled, isDaVinciClient, isDaVinciCurrentVersionBootstrappingSpeedupEnabled, isDebugLoggingEnabled, isDedicatedConsumerPoolForAAWCLeaderEnabled, isDedicatedDrainerQueueEnabled, isDeleteUnassignedPartitionsOnStartupEnabled, isDiskHealthCheckServiceEnabled, isEnableParallelBatchGet, isGlobalRtDivEnabled, isGrpcEnabled, isHelixHybridStoreQuotaEnabled, isHelixJoinAsUnknownEnabled, isHttp2InboundEnabled, isHybridQuotaEnabled, isIngestionMlockEnabled, isKafkaConsumerOffsetCollectionEnabled, isKeyValueProfilingEnabled, isKMERegistrationFromMessageHeaderEnabled, isLeaderCompleteStateCheckInFollowerEnabled, isLeakedResourceCleanupEnabled, isLiveConfigBasedKafkaThrottlingEnabled, isNearlineWorkloadProducerThroughputOptimizationEnabled, isOptimizeDatabaseForBackupVersionEnabled, isQuotaEnforcementEnabled, isReadOnlyForBatchOnlyStoreEnabled, isRecordLevelMetricWhenBootstrappingCurrentVersionEnabled, isResetErrorReplicaEnabled, isRestServiceEpollEnabled, isResubscriptionTriggeredByVersionIngestionContextChangeEnabled, isRocksDbStorageEngineConfigCheckEnabled, isSchemaPresenceCheckEnabled, isServerAllowlistEnabled, isServerCalculateQuotaUsageBasedOnPartitionsAssignmentEnabled, isServerIngestionCheckpointDuringGracefulShutdownEnabled, isStoreWriterBufferAfterLeaderLogicEnabled, isStuckConsumerRepairEnabled, isSystemSchemaInitializationAtStartTimeEnabled, isUnregisterMetricForDeletedStoreEnabled, isUnsubscribeAfterBatchpushEnabled, useDaVinciSpecificExecutionStatusForError

Methods inherited from class com.linkedin.davinci.config.VeniceClusterConfig

getClusterName, getClusterProperties, getEquivalentKafkaClusterIdForSepTopic, getKafkaBootstrapServers, getKafkaClusterAliasToIdMap, getKafkaClusterIdToAliasMap, getKafkaClusterIdToUrlMap, getKafkaClusterMap, getKafkaClusterUrlResolver, getKafkaClusterUrlToAliasMap, getKafkaClusterUrlToIdMap, getKafkaEmptyPollSleepMs, getKafkaFetchMaxSizePerSecond, getKafkaFetchMaxTimeMS, getKafkaFetchMinSizePerSecond, getKafkaFetchPartitionMaxSizePerSecond, getKafkaFetchQuotaBytesPerSecond, getKafkaFetchQuotaRecordPerSecond, getKafkaFetchQuotaTimeWindow, getKafkaFetchQuotaUnorderedBytesPerSecond, getKafkaFetchQuotaUnorderedRecordPerSecond, getKafkaReadCycleDelayMs, getKafkaSecurityProtocol, getPersistenceType, getRefreshAttemptsForZkReconnect, getRefreshIntervalForZkReconnectInMs, getRegionName, getRegionNames, getSslConfig, getZookeeperAddress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VeniceStoreVersionConfig

public VeniceStoreVersionConfig(String storeVersionName,
 VeniceProperties storeProperties,
 Map<String,Map<String,String>> kafkaClusterMap)

VeniceStoreVersionConfig

public VeniceStoreVersionConfig(String storeVersionName,
 VeniceProperties storeProperties)
                         throws ConfigurationException

Throws:

ConfigurationException

VeniceStoreVersionConfig

public VeniceStoreVersionConfig(String storeVersionName,
 VeniceProperties storeProperties,
 PersistenceType storePersistenceType)
                         throws ConfigurationException

Throws:

ConfigurationException

Method Details

getStoreVersionName

public String getStoreVersionName()

isBlobTransferEnabled

public boolean isBlobTransferEnabled()

setBlobTransferEnabled

public void setBlobTransferEnabled(boolean blobTransferEnabled)

getStorePersistenceType

public PersistenceType getStorePersistenceType()

isRestoreDataPartitions

public boolean isRestoreDataPartitions()

setRestoreDataPartitions

public void setRestoreDataPartitions(boolean restoreDataPartitions)

isRestoreMetadataPartition

public boolean isRestoreMetadataPartition()

setRestoreMetadataPartition

public void setRestoreMetadataPartition(boolean restoreMetadataPartition)

isStorePersistenceTypeKnown

public boolean isStorePersistenceTypeKnown()

For some store, the persistence type may not be known when constructing VeniceStoreVersionConfig, such as in `VeniceStateModelFactory#createNewStateModel`, when Helix wants to create a new state model for some store, it doesn't know the persistence type since it is possible that this store is an existing store, so the persistence type is decided by the on-disk type, and it is a new store, it will be decided by storage node config.

Returns:

true, it means the storage persistence type is decided, and stay immutable during the lifetime of this store in the same node.

setStorePersistenceType

public void setStorePersistenceType(PersistenceType storePersistenceType)

This reason to create a `setter` for storePersistenceType since it is unknown sometimes when initializing VeniceStoreVersionConfig, such as in com.linkedin.venice.helix.VeniceStateModelFactory#createNewStateModel, The new state model could be created for a brand-new store or an existing store.

Parameters:

storePersistenceType -

getPersistStorageEngineConfig

public VeniceProperties getPersistStorageEngineConfig()