Class ZKStore
- java.lang.Object
-
- com.linkedin.venice.meta.AbstractStore
-
- com.linkedin.venice.meta.ZKStore
-
- All Implemented Interfaces:
DataModelBackedStructure<StoreProperties>
,Store
public class ZKStore extends AbstractStore implements DataModelBackedStructure<StoreProperties>
Class defines the store of Venice.This class is NOT thread safe. Concurrency request to ZKStore instance should be controlled in repository level. When adding fields to Store Metadata (stored in ZK or metadata system store), you need to modify avro schema in this folder: venice-common/src/main/resources/avro/StoreMeta/StoreMetaValue Before we enable the zk shared metadata system schema store auto creation:
ConfigKeys.CONTROLLER_ZK_SHARED_META_SYSTEM_SCHEMA_STORE_AUTO_CREATION_ENABLED
, we could continue to modify v1 schema. But once this is enabled, we need to evolve the value schema by adding a new version, and DON'T EVER CHANGE THE EXISTING SCHEMA WITHOUT CAREFUL CONSIDERATION!!! Also make sure json serialization still works. So the steps will become the following when you want to add a new field to Store Metadata: 1. Evolve the value schema of metadata system schema mentioned by the above section, and always have a default for the newly added fields. 2. Add getter/setter methods toStore
. When you want to add a simple field to Store metadata, you just need to create getter/setter for the new field. When you try to add a method starting with 'get', the default json serialization will do serialization by this method, which could produce some unexpected serialization result, so if it is not for serialization purpose, please specifyJsonIgnore
to ignore the method, whose name is starting with 'get'. TODO: we need to refactor this class to separate Store operations from Store POJO, which is being used by JSON TODO: Since metadata keeps increasing, maybe we would like to refactor it to builder pattern. TODO: In the future, we could consider to use avro json serialization directly to make it simpler.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class com.linkedin.venice.meta.AbstractStore
AbstractStore.StoreVersionSupplier
-
-
Field Summary
-
Fields inherited from class com.linkedin.venice.meta.AbstractStore
DEFAULT_READ_QUOTA, DEFAULT_REPLICATION_FACTOR, DEFAULT_STORAGE_QUOTA
-
Fields inherited from interface com.linkedin.venice.meta.Store
BOOTSTRAP_TO_ONLINE_TIMEOUT_IN_HOURS, DEFAULT_BATCH_GET_LIMIT, DEFAULT_RT_RETENTION_TIME, IGNORE_VERSION, NON_EXISTING_VERSION, NUM_VERSION_PRESERVE_NOT_SET, storeNamePattern, SYSTEM_STORE_FORMAT, SYSTEM_STORE_NAME_PREFIX, UNLIMITED_STORAGE_QUOTA
-
-
Constructor Summary
Constructors Constructor Description ZKStore(Store store)
ZKStore(StoreProperties storeProperties)
ZKStore(java.lang.String name, java.lang.String owner, long createdTimeMs, PersistenceType persistenceType, RoutingStrategy routingStrategy, ReadStrategy readStrategy, OfflinePushStrategy offlinePushStrategy, int replicationFactor)
ZKStore(java.lang.String name, java.lang.String owner, long createdTime, PersistenceType persistenceType, RoutingStrategy routingStrategy, ReadStrategy readStrategy, OfflinePushStrategy offlinePushStrategy, int currentVersion, long storageQuotaInByte, long readQuotaInCU, HybridStoreConfig hybridStoreConfig, PartitionerConfig partitionerConfig, int replicationFactor)
-
Method Summary
-
Methods inherited from class com.linkedin.venice.meta.AbstractStore
addVersion, addVersion, checkDisableStoreWrite, containsVersion, deleteVersion, fixMissingFields, forceAddVersion, getVersion, getVersionOrThrow, getVersions, getVersionStatus, isSystemStore, peekNextVersion, retrieveVersionsToDelete, setupVersionSupplier, updateVersionStatus
-
-
-
-
Constructor Detail
-
ZKStore
public ZKStore(java.lang.String name, java.lang.String owner, long createdTimeMs, PersistenceType persistenceType, RoutingStrategy routingStrategy, ReadStrategy readStrategy, OfflinePushStrategy offlinePushStrategy, int replicationFactor)
-
ZKStore
public ZKStore(java.lang.String name, java.lang.String owner, long createdTime, PersistenceType persistenceType, RoutingStrategy routingStrategy, ReadStrategy readStrategy, OfflinePushStrategy offlinePushStrategy, int currentVersion, long storageQuotaInByte, long readQuotaInCU, HybridStoreConfig hybridStoreConfig, PartitionerConfig partitionerConfig, int replicationFactor)
-
ZKStore
public ZKStore(StoreProperties storeProperties)
-
ZKStore
public ZKStore(Store store)
-
-
Method Detail
-
dataModel
public StoreProperties dataModel()
Description copied from interface:DataModelBackedStructure
Return the backed data model.- Specified by:
dataModel
in interfaceDataModelBackedStructure<StoreProperties>
- Returns:
-
getCreatedTime
public long getCreatedTime()
- Specified by:
getCreatedTime
in interfaceStore
-
getCurrentVersion
public int getCurrentVersion()
- Specified by:
getCurrentVersion
in interfaceStore
-
setCurrentVersion
public void setCurrentVersion(int currentVersion)
Set current serving version number of this store. If store is disabled to write, thrownStoreDisabledException
.- Specified by:
setCurrentVersion
in interfaceStore
-
setCurrentVersionWithoutCheck
public void setCurrentVersionWithoutCheck(int currentVersion)
- Specified by:
setCurrentVersionWithoutCheck
in interfaceStore
-
getLowWatermark
public long getLowWatermark()
- Specified by:
getLowWatermark
in interfaceStore
-
setLowWatermark
public void setLowWatermark(long lowWatermark)
- Specified by:
setLowWatermark
in interfaceStore
-
getPersistenceType
public PersistenceType getPersistenceType()
- Specified by:
getPersistenceType
in interfaceStore
-
setPersistenceType
public void setPersistenceType(PersistenceType persistenceType)
- Specified by:
setPersistenceType
in interfaceStore
-
getRoutingStrategy
public RoutingStrategy getRoutingStrategy()
- Specified by:
getRoutingStrategy
in interfaceStore
-
getReadStrategy
public ReadStrategy getReadStrategy()
- Specified by:
getReadStrategy
in interfaceStore
-
getOffLinePushStrategy
public OfflinePushStrategy getOffLinePushStrategy()
- Specified by:
getOffLinePushStrategy
in interfaceStore
-
setVersions
public void setVersions(java.util.List<Version> versions)
- Specified by:
setVersions
in interfaceStore
- Overrides:
setVersions
in classAbstractStore
-
getLargestUsedVersionNumber
public int getLargestUsedVersionNumber()
- Specified by:
getLargestUsedVersionNumber
in interfaceStore
-
setLargestUsedVersionNumber
public void setLargestUsedVersionNumber(int largestUsedVersionNumber)
- Specified by:
setLargestUsedVersionNumber
in interfaceStore
-
getStorageQuotaInByte
public long getStorageQuotaInByte()
- Specified by:
getStorageQuotaInByte
in interfaceStore
-
setStorageQuotaInByte
public void setStorageQuotaInByte(long storageQuotaInByte)
- Specified by:
setStorageQuotaInByte
in interfaceStore
-
getPartitionCount
public int getPartitionCount()
- Specified by:
getPartitionCount
in interfaceStore
-
setPartitionCount
public void setPartitionCount(int partitionCount)
- Specified by:
setPartitionCount
in interfaceStore
-
getPartitionerConfig
public PartitionerConfig getPartitionerConfig()
- Specified by:
getPartitionerConfig
in interfaceStore
-
setPartitionerConfig
public void setPartitionerConfig(PartitionerConfig value)
- Specified by:
setPartitionerConfig
in interfaceStore
-
isEnableWrites
public boolean isEnableWrites()
- Specified by:
isEnableWrites
in interfaceStore
-
setEnableWrites
public void setEnableWrites(boolean enableWrites)
- Specified by:
setEnableWrites
in interfaceStore
-
isEnableReads
public boolean isEnableReads()
- Specified by:
isEnableReads
in interfaceStore
-
setEnableReads
public void setEnableReads(boolean enableReads)
- Specified by:
setEnableReads
in interfaceStore
-
getReadQuotaInCU
public long getReadQuotaInCU()
- Specified by:
getReadQuotaInCU
in interfaceStore
-
setReadQuotaInCU
public void setReadQuotaInCU(long readQuotaInCU)
- Specified by:
setReadQuotaInCU
in interfaceStore
-
getHybridStoreConfig
public HybridStoreConfig getHybridStoreConfig()
- Specified by:
getHybridStoreConfig
in interfaceStore
-
setHybridStoreConfig
public void setHybridStoreConfig(HybridStoreConfig hybridStoreConfig)
- Specified by:
setHybridStoreConfig
in interfaceStore
-
getViewConfigs
public java.util.Map<java.lang.String,ViewConfig> getViewConfigs()
- Specified by:
getViewConfigs
in interfaceStore
-
setViewConfigs
public void setViewConfigs(java.util.Map<java.lang.String,ViewConfig> viewConfigList)
- Specified by:
setViewConfigs
in interfaceStore
-
getCompressionStrategy
public CompressionStrategy getCompressionStrategy()
- Specified by:
getCompressionStrategy
in interfaceStore
-
setCompressionStrategy
public void setCompressionStrategy(CompressionStrategy compressionStrategy)
- Specified by:
setCompressionStrategy
in interfaceStore
-
getClientDecompressionEnabled
public boolean getClientDecompressionEnabled()
- Specified by:
getClientDecompressionEnabled
in interfaceStore
-
setClientDecompressionEnabled
public void setClientDecompressionEnabled(boolean clientDecompressionEnabled)
- Specified by:
setClientDecompressionEnabled
in interfaceStore
-
isChunkingEnabled
public boolean isChunkingEnabled()
- Specified by:
isChunkingEnabled
in interfaceStore
-
setChunkingEnabled
public void setChunkingEnabled(boolean chunkingEnabled)
- Specified by:
setChunkingEnabled
in interfaceStore
-
isRmdChunkingEnabled
public boolean isRmdChunkingEnabled()
- Specified by:
isRmdChunkingEnabled
in interfaceStore
-
setRmdChunkingEnabled
public void setRmdChunkingEnabled(boolean rmdChunkingEnabled)
- Specified by:
setRmdChunkingEnabled
in interfaceStore
-
getBatchGetLimit
public int getBatchGetLimit()
- Specified by:
getBatchGetLimit
in interfaceStore
-
setBatchGetLimit
public void setBatchGetLimit(int batchGetLimit)
- Specified by:
setBatchGetLimit
in interfaceStore
-
isIncrementalPushEnabled
public boolean isIncrementalPushEnabled()
- Specified by:
isIncrementalPushEnabled
in interfaceStore
-
setIncrementalPushEnabled
public void setIncrementalPushEnabled(boolean incrementalPushEnabled)
- Specified by:
setIncrementalPushEnabled
in interfaceStore
-
isSeparateRealTimeTopicEnabled
public boolean isSeparateRealTimeTopicEnabled()
- Specified by:
isSeparateRealTimeTopicEnabled
in interfaceStore
-
setSeparateRealTimeTopicEnabled
public void setSeparateRealTimeTopicEnabled(boolean separateRealTimeTopicEnabled)
- Specified by:
setSeparateRealTimeTopicEnabled
in interfaceStore
-
isAccessControlled
public boolean isAccessControlled()
Deprecated.The store level accessControlled flag is no longer valid to be used to skip ACL checks.- Specified by:
isAccessControlled
in interfaceStore
-
setAccessControlled
public void setAccessControlled(boolean accessControlled)
Deprecated.The store level accessControlled flag is no longer valid to be used to skip ACL checks.- Specified by:
setAccessControlled
in interfaceStore
-
isMigrating
public boolean isMigrating()
- Specified by:
isMigrating
in interfaceStore
-
setMigrating
public void setMigrating(boolean migrating)
- Specified by:
setMigrating
in interfaceStore
-
getNumVersionsToPreserve
public int getNumVersionsToPreserve()
- Specified by:
getNumVersionsToPreserve
in interfaceStore
-
setNumVersionsToPreserve
public void setNumVersionsToPreserve(int numVersionsToPreserve)
- Specified by:
setNumVersionsToPreserve
in interfaceStore
-
isWriteComputationEnabled
public boolean isWriteComputationEnabled()
- Specified by:
isWriteComputationEnabled
in interfaceStore
-
setWriteComputationEnabled
public void setWriteComputationEnabled(boolean writeComputationEnabled)
- Specified by:
setWriteComputationEnabled
in interfaceStore
-
isReadComputationEnabled
public boolean isReadComputationEnabled()
- Specified by:
isReadComputationEnabled
in interfaceStore
-
setReadComputationEnabled
public void setReadComputationEnabled(boolean readComputationEnabled)
- Specified by:
setReadComputationEnabled
in interfaceStore
-
getBootstrapToOnlineTimeoutInHours
public int getBootstrapToOnlineTimeoutInHours()
- Specified by:
getBootstrapToOnlineTimeoutInHours
in interfaceStore
-
setBootstrapToOnlineTimeoutInHours
public void setBootstrapToOnlineTimeoutInHours(int bootstrapToOnlineTimeoutInHours)
- Specified by:
setBootstrapToOnlineTimeoutInHours
in interfaceStore
-
getPushStreamSourceAddress
public java.lang.String getPushStreamSourceAddress()
- Specified by:
getPushStreamSourceAddress
in interfaceStore
-
setPushStreamSourceAddress
public void setPushStreamSourceAddress(java.lang.String sourceAddress)
- Specified by:
setPushStreamSourceAddress
in interfaceStore
-
isNativeReplicationEnabled
public boolean isNativeReplicationEnabled()
- Specified by:
isNativeReplicationEnabled
in interfaceStore
-
getRmdVersion
public int getRmdVersion()
- Specified by:
getRmdVersion
in interfaceStore
-
setRmdVersion
public void setRmdVersion(int rmdVersion)
- Specified by:
setRmdVersion
in interfaceStore
-
setNativeReplicationEnabled
public void setNativeReplicationEnabled(boolean nativeReplicationEnabled)
- Specified by:
setNativeReplicationEnabled
in interfaceStore
-
getBackupStrategy
public BackupStrategy getBackupStrategy()
- Specified by:
getBackupStrategy
in interfaceStore
-
setBackupStrategy
public void setBackupStrategy(BackupStrategy value)
- Specified by:
setBackupStrategy
in interfaceStore
-
isSchemaAutoRegisterFromPushJobEnabled
public boolean isSchemaAutoRegisterFromPushJobEnabled()
- Specified by:
isSchemaAutoRegisterFromPushJobEnabled
in interfaceStore
-
setSchemaAutoRegisterFromPushJobEnabled
public void setSchemaAutoRegisterFromPushJobEnabled(boolean value)
- Specified by:
setSchemaAutoRegisterFromPushJobEnabled
in interfaceStore
-
getLatestSuperSetValueSchemaId
public int getLatestSuperSetValueSchemaId()
- Specified by:
getLatestSuperSetValueSchemaId
in interfaceStore
-
setLatestSuperSetValueSchemaId
public void setLatestSuperSetValueSchemaId(int valueSchemaId)
- Specified by:
setLatestSuperSetValueSchemaId
in interfaceStore
-
isHybridStoreDiskQuotaEnabled
public boolean isHybridStoreDiskQuotaEnabled()
- Specified by:
isHybridStoreDiskQuotaEnabled
in interfaceStore
-
setHybridStoreDiskQuotaEnabled
public void setHybridStoreDiskQuotaEnabled(boolean enabled)
- Specified by:
setHybridStoreDiskQuotaEnabled
in interfaceStore
-
getEtlStoreConfig
public ETLStoreConfig getEtlStoreConfig()
- Specified by:
getEtlStoreConfig
in interfaceStore
-
setEtlStoreConfig
public void setEtlStoreConfig(ETLStoreConfig etlStoreConfig)
- Specified by:
setEtlStoreConfig
in interfaceStore
-
isStoreMetadataSystemStoreEnabled
public boolean isStoreMetadataSystemStoreEnabled()
- Specified by:
isStoreMetadataSystemStoreEnabled
in interfaceStore
-
setStoreMetadataSystemStoreEnabled
public void setStoreMetadataSystemStoreEnabled(boolean storeMetadataSystemStoreEnabled)
- Specified by:
setStoreMetadataSystemStoreEnabled
in interfaceStore
-
isStoreMetaSystemStoreEnabled
public boolean isStoreMetaSystemStoreEnabled()
- Specified by:
isStoreMetaSystemStoreEnabled
in interfaceStore
-
setStoreMetaSystemStoreEnabled
public void setStoreMetaSystemStoreEnabled(boolean storeMetaSystemStoreEnabled)
- Specified by:
setStoreMetaSystemStoreEnabled
in interfaceStore
-
getLatestVersionPromoteToCurrentTimestamp
public long getLatestVersionPromoteToCurrentTimestamp()
- Specified by:
getLatestVersionPromoteToCurrentTimestamp
in interfaceStore
-
setLatestVersionPromoteToCurrentTimestamp
public void setLatestVersionPromoteToCurrentTimestamp(long latestVersionPromoteToCurrentTimestamp)
- Specified by:
setLatestVersionPromoteToCurrentTimestamp
in interfaceStore
-
getBackupVersionRetentionMs
public long getBackupVersionRetentionMs()
- Specified by:
getBackupVersionRetentionMs
in interfaceStore
-
setBackupVersionRetentionMs
public void setBackupVersionRetentionMs(long backupVersionRetentionMs)
- Specified by:
setBackupVersionRetentionMs
in interfaceStore
-
getRetentionTime
public long getRetentionTime()
Get the retention time for the RT Topic. If there exists a HybridStoreConfig, return the retention time from the config file. Otherwise, if write compute is enabled, then return the default retention time.- Specified by:
getRetentionTime
in interfaceStore
- Returns:
- the retention time for the RT topic, in milliseconds.
-
getReplicationFactor
public int getReplicationFactor()
- Specified by:
getReplicationFactor
in interfaceStore
-
setReplicationFactor
public void setReplicationFactor(int replicationFactor)
- Specified by:
setReplicationFactor
in interfaceStore
-
isMigrationDuplicateStore
public boolean isMigrationDuplicateStore()
- Specified by:
isMigrationDuplicateStore
in interfaceStore
-
setMigrationDuplicateStore
public void setMigrationDuplicateStore(boolean migrationDuplicateStore)
- Specified by:
setMigrationDuplicateStore
in interfaceStore
-
getNativeReplicationSourceFabric
public java.lang.String getNativeReplicationSourceFabric()
- Specified by:
getNativeReplicationSourceFabric
in interfaceStore
-
setNativeReplicationSourceFabric
public void setNativeReplicationSourceFabric(java.lang.String nativeReplicationSourceFabric)
- Specified by:
setNativeReplicationSourceFabric
in interfaceStore
-
isActiveActiveReplicationEnabled
public boolean isActiveActiveReplicationEnabled()
- Specified by:
isActiveActiveReplicationEnabled
in interfaceStore
-
setActiveActiveReplicationEnabled
public void setActiveActiveReplicationEnabled(boolean activeActiveReplicationEnabled)
- Specified by:
setActiveActiveReplicationEnabled
in interfaceStore
-
getSystemStores
public java.util.Map<java.lang.String,SystemStoreAttributes> getSystemStores()
- Specified by:
getSystemStores
in interfaceStore
-
setSystemStores
public void setSystemStores(java.util.Map<java.lang.String,SystemStoreAttributes> systemStores)
- Specified by:
setSystemStores
in interfaceStore
-
putSystemStore
public void putSystemStore(VeniceSystemStoreType systemStoreType, SystemStoreAttributes systemStoreAttributes)
- Specified by:
putSystemStore
in interfaceStore
-
isDaVinciPushStatusStoreEnabled
public boolean isDaVinciPushStatusStoreEnabled()
- Specified by:
isDaVinciPushStatusStoreEnabled
in interfaceStore
-
setDaVinciPushStatusStoreEnabled
public void setDaVinciPushStatusStoreEnabled(boolean daVinciPushStatusStoreEnabled)
- Specified by:
setDaVinciPushStatusStoreEnabled
in interfaceStore
-
isStorageNodeReadQuotaEnabled
public boolean isStorageNodeReadQuotaEnabled()
- Specified by:
isStorageNodeReadQuotaEnabled
in interfaceStore
-
setStorageNodeReadQuotaEnabled
public void setStorageNodeReadQuotaEnabled(boolean storageNodeReadQuotaEnabled)
- Specified by:
setStorageNodeReadQuotaEnabled
in interfaceStore
-
getMinCompactionLagSeconds
public long getMinCompactionLagSeconds()
- Specified by:
getMinCompactionLagSeconds
in interfaceStore
-
setMinCompactionLagSeconds
public void setMinCompactionLagSeconds(long minCompactionLagSeconds)
- Specified by:
setMinCompactionLagSeconds
in interfaceStore
-
getMaxCompactionLagSeconds
public long getMaxCompactionLagSeconds()
- Specified by:
getMaxCompactionLagSeconds
in interfaceStore
-
setMaxCompactionLagSeconds
public void setMaxCompactionLagSeconds(long maxCompactionLagSeconds)
- Specified by:
setMaxCompactionLagSeconds
in interfaceStore
-
getMaxRecordSizeBytes
public int getMaxRecordSizeBytes()
- Specified by:
getMaxRecordSizeBytes
in interfaceStore
-
setMaxRecordSizeBytes
public void setMaxRecordSizeBytes(int maxRecordSizeBytes)
- Specified by:
setMaxRecordSizeBytes
in interfaceStore
-
getMaxNearlineRecordSizeBytes
public int getMaxNearlineRecordSizeBytes()
- Specified by:
getMaxNearlineRecordSizeBytes
in interfaceStore
-
setMaxNearlineRecordSizeBytes
public void setMaxNearlineRecordSizeBytes(int maxNearlineRecordSizeBytes)
- Specified by:
setMaxNearlineRecordSizeBytes
in interfaceStore
-
setUnusedSchemaDeletionEnabled
public void setUnusedSchemaDeletionEnabled(boolean unusedSchemaDeletionEnabled)
- Specified by:
setUnusedSchemaDeletionEnabled
in interfaceStore
-
isUnusedSchemaDeletionEnabled
public boolean isUnusedSchemaDeletionEnabled()
- Specified by:
isUnusedSchemaDeletionEnabled
in interfaceStore
-
setBlobTransferEnabled
public void setBlobTransferEnabled(boolean blobTransferEnabled)
- Specified by:
setBlobTransferEnabled
in interfaceStore
-
isBlobTransferEnabled
public boolean isBlobTransferEnabled()
- Specified by:
isBlobTransferEnabled
in interfaceStore
-
isNearlineProducerCompressionEnabled
public boolean isNearlineProducerCompressionEnabled()
- Specified by:
isNearlineProducerCompressionEnabled
in interfaceStore
-
setNearlineProducerCompressionEnabled
public void setNearlineProducerCompressionEnabled(boolean compressionEnabled)
- Specified by:
setNearlineProducerCompressionEnabled
in interfaceStore
-
getNearlineProducerCountPerWriter
public int getNearlineProducerCountPerWriter()
- Specified by:
getNearlineProducerCountPerWriter
in interfaceStore
-
setNearlineProducerCountPerWriter
public void setNearlineProducerCountPerWriter(int producerCnt)
- Specified by:
setNearlineProducerCountPerWriter
in interfaceStore
-
equals
public boolean equals(java.lang.Object o)
- Overrides:
equals
in classjava.lang.Object
-
hashCode
public int hashCode()
- Overrides:
hashCode
in classjava.lang.Object
-
cloneStore
public Store cloneStore()
Cloned a new store based on current data in this store. TODO: once the whole stack (all the users of this class) migrates to use modern avro version (1.7+), we could useGenericData.deepCopy(org.apache.avro.Schema, T)
to do clone the internal data model:storeProperties
, which will be more convenient.- Specified by:
cloneStore
in interfaceStore
- Returns:
- cloned store.
-
-