Package com.linkedin.venice.meta

Class AbstractStore

java.lang.Object

com.linkedin.venice.meta.AbstractStore

All Implemented Interfaces:

Store

Direct Known Subclasses:

SystemStore, ZKStore

public abstract class AbstractStore
extends Object
implements Store

This is an abstraction of metadata maintained per Store.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static interface	AbstractStore.StoreVersionSupplier

Field Summary

Fields

Modifier and Type	Field	Description
static final long	DEFAULT_READ_QUOTA	Default read quota 1800 QPS per node
static final int	DEFAULT_REPLICATION_FACTOR
static final long	DEFAULT_STORAGE_QUOTA	Default storage quota 20GB

Fields inherited from interface com.linkedin.venice.meta.Store

BOOTSTRAP_TO_ONLINE_TIMEOUT_IN_HOURS, DEFAULT_BATCH_GET_LIMIT, DEFAULT_RT_RETENTION_TIME, IGNORE_VERSION, NON_EXISTING_VERSION, NUM_VERSION_PRESERVE_NOT_SET, SYSTEM_STORE_FORMAT, SYSTEM_STORE_NAME_PREFIX, UNLIMITED_STORAGE_QUOTA

Constructor Summary

Constructors

Constructor	Description
AbstractStore()

Method Summary

Modifier and Type	Method	Description
void	addVersion(Version version)
void	addVersion(Version version, boolean isClonedVersion, int currentRTVersionNumber)
void	checkDisableStoreWrite(String action, int version)
static List<Version>	computeVersionsToDelete(List<Version> versions, int currentVersion, int clusterNumVersionsToPreserve, int storeNumVersionsToPreserve, boolean isMigrating)	Same version-deletion policy as retrieveVersionsToDelete(int), but over raw fields so callers that only hold a version snapshot (e.g.
boolean	containsVersion(int versionNumber)
Version	deleteVersion(int versionNumber)
void	fixMissingFields()
void	forceAddVersion(Version version, boolean isClonedVersion)
Version	getVersion(int versionNumber)
Version	getVersionOrThrow(int versionNumber)
List<Version>	getVersions()
VersionStatus	getVersionStatus(int versionNumber)
boolean	isSystemStore()
int	peekNextVersionNumber()
List<Version>	retrieveVersionsToDelete(int clusterNumVersionsToPreserve)
protected void	setupVersionSupplier(AbstractStore.StoreVersionSupplier versionsSupplier)	This function should be invoked only once.
void	setVersions(List<Version> versions)
void	setVersionTargetRegionPromoted(int versionNumber, boolean targetRegionPromoted)
void	updateVersionForDaVinciHeartbeat(int versionNumber, boolean reported)
void	updateVersionStatus(int versionNumber, VersionStatus status)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.venice.meta.Store

cloneStore, getBackupStrategy, getBackupVersionRetentionMs, getBatchGetLimit, getBlobDbEnabled, getBlobTransferInServerEnabled, getBootstrapToOnlineTimeoutInHours, getClientDecompressionEnabled, getCompactionThresholdMilliseconds, getCompressionStrategy, getCreatedTime, getCurrentVersion, getEtlStoreConfig, getExternalStorageReadMode, getHybridStoreConfig, getIngestionPausedRegions, getIngestionPauseMode, getIsDavinciHeartbeatReported, getKeyUrnFields, getLargestUsedRTVersionNumber, getLargestUsedVersionNumber, getLatestSuperSetValueSchemaId, getLatestVersionPromoteToCurrentTimestamp, getLowWatermark, getMaxCompactionLagSeconds, getMaxNearlineRecordSizeBytes, getMaxRecordSizeBytes, getMinCompactionLagSeconds, getName, getNativeReplicationSourceFabric, getNearlineProducerCountPerWriter, getNumVersionsToPreserve, getOffLinePushStrategy, getOwner, getPartitionCount, getPartitionerConfig, getPersistenceType, getPreviousCurrentVersion, getPushStreamSourceAddress, getReadQuotaInCU, getReadStrategy, getReplicationFactor, getRetentionTime, getRmdVersion, getRoutingStrategy, getStorageMode, getStorageQuotaInByte, getStoreLifecycleHooks, getSystemStores, getTargetSwapRegion, getTargetSwapRegionWaitTime, getVersionNumbers, getViewConfigs, isAccessControlled, isActiveActiveReplicationEnabled, isBlobTransferEnabled, isChunkingEnabled, isCompactionEnabled, isDaVinciPushStatusStoreEnabled, isEnableReads, isEnableWrites, isEncryptionEnabled, isEnumSchemaEvolutionAllowed, isFlinkVeniceViewsEnabled, isGlobalRtDivEnabled, isHybrid, isHybridStoreDiskQuotaEnabled, isIncrementalPushEnabled, isKeyUrnCompressionEnabled, isMigrating, isMigrationDuplicateStore, isNativeReplicationEnabled, isNearlineProducerCompressionEnabled, isReadComputationEnabled, isRmdChunkingEnabled, isSchemaAutoRegisterFromPushJobEnabled, isSeparateRealTimeTopicEnabled, isStorageNodeReadQuotaEnabled, isStoreMetadataSystemStoreEnabled, isStoreMetaSystemStoreEnabled, isTTLRepushEnabled, isUnusedSchemaDeletionEnabled, isWriteComputationEnabled, putSystemStore, setAccessControlled, setActiveActiveReplicationEnabled, setBackupStrategy, setBackupVersionRetentionMs, setBatchGetLimit, setBlobDbEnabled, setBlobTransferEnabled, setBlobTransferInServerEnabled, setBootstrapToOnlineTimeoutInHours, setChunkingEnabled, setClientDecompressionEnabled, setCompactionEnabled, setCompactionThresholdMilliseconds, setCompressionStrategy, setCurrentVersion, setCurrentVersionWithoutCheck, setDaVinciPushStatusStoreEnabled, setEnableReads, setEnableWrites, setEncryptionEnabled, setEnumSchemaEvolutionAllowed, setEtlStoreConfig, setExternalStorageReadMode, setFlinkVeniceViewsEnabled, setGlobalRtDivEnabled, setHybridStoreConfig, setHybridStoreDiskQuotaEnabled, setIncrementalPushEnabled, setIngestionPausedRegions, setIngestionPauseMode, setIsDavinciHeartbeatReported, setKeyUrnCompressionEnabled, setKeyUrnFields, setLargestUsedRTVersionNumber, setLargestUsedVersionNumber, setLatestSuperSetValueSchemaId, setLatestVersionPromoteToCurrentTimestamp, setLowWatermark, setMaxCompactionLagSeconds, setMaxNearlineRecordSizeBytes, setMaxRecordSizeBytes, setMigrating, setMigrationDuplicateStore, setMinCompactionLagSeconds, setNativeReplicationEnabled, setNativeReplicationSourceFabric, setNearlineProducerCompressionEnabled, setNearlineProducerCountPerWriter, setNumVersionsToPreserve, setOwner, setPartitionCount, setPartitionerConfig, setPersistenceType, setPreviousCurrentVersion, setPushStreamSourceAddress, setReadComputationEnabled, setReadQuotaInCU, setReplicationFactor, setRmdChunkingEnabled, setRmdVersion, setSchemaAutoRegisterFromPushJobEnabled, setSeparateRealTimeTopicEnabled, setStorageMode, setStorageNodeReadQuotaEnabled, setStorageQuotaInByte, setStoreLifecycleHooks, setStoreMetadataSystemStoreEnabled, setStoreMetaSystemStoreEnabled, setSystemStores, setTargetSwapRegion, setTargetSwapRegionWaitTime, setTTLRepushEnabled, setUnusedSchemaDeletionEnabled, setViewConfigs, setWriteComputationEnabled

Field Details

DEFAULT_REPLICATION_FACTOR

public static final int DEFAULT_REPLICATION_FACTOR

See Also:

• Constant Field Values

DEFAULT_STORAGE_QUOTA

public static final long DEFAULT_STORAGE_QUOTA

Default storage quota 20GB

See Also:

• Constant Field Values

DEFAULT_READ_QUOTA

public static final long DEFAULT_READ_QUOTA

Default read quota 1800 QPS per node

See Also:

• Constant Field Values

Constructor Details

AbstractStore

public AbstractStore()

Method Details

setupVersionSupplier

protected void setupVersionSupplier(AbstractStore.StoreVersionSupplier versionsSupplier)

This function should be invoked only once.

getVersions

public List<Version> getVersions()

Specified by:

getVersions in interface Store

setVersions

public void setVersions(List<Version> versions)

Specified by:

setVersions in interface Store

addVersion

public void addVersion(Version version)

Specified by:

addVersion in interface Store

addVersion

public void addVersion(Version version,
 boolean isClonedVersion,
 int currentRTVersionNumber)

Specified by:

addVersion in interface Store

forceAddVersion

public void forceAddVersion(Version version,
 boolean isClonedVersion)

Specified by:

forceAddVersion in interface Store

checkDisableStoreWrite

public void checkDisableStoreWrite(String action,
 int version)

Specified by:

checkDisableStoreWrite in interface Store

deleteVersion

public Version deleteVersion(int versionNumber)

Specified by:

deleteVersion in interface Store

containsVersion

public boolean containsVersion(int versionNumber)

Specified by:

containsVersion in interface Store

updateVersionStatus

public void updateVersionStatus(int versionNumber,
 VersionStatus status)

Specified by:

updateVersionStatus in interface Store

setVersionTargetRegionPromoted

public void setVersionTargetRegionPromoted(int versionNumber,
 boolean targetRegionPromoted)

Specified by:

setVersionTargetRegionPromoted in interface Store

peekNextVersionNumber

public int peekNextVersionNumber()

Specified by:

peekNextVersionNumber in interface Store

getVersion

@Nullable
public Version getVersion(int versionNumber)

Specified by:

getVersion in interface Store

Parameters:

versionNumber - for which to get the Version

Returns:

the Version corresponding to the provided , or null if no such version exists

getVersionOrThrow

@Nonnull
public Version getVersionOrThrow(int versionNumber)
                          throws StoreVersionNotFoundException

Specified by:

getVersionOrThrow in interface Store

Throws:

StoreVersionNotFoundException

getVersionStatus

public VersionStatus getVersionStatus(int versionNumber)

Specified by:

getVersionStatus in interface Store

retrieveVersionsToDelete

public List<Version> retrieveVersionsToDelete(int clusterNumVersionsToPreserve)

Specified by:

retrieveVersionsToDelete in interface Store

computeVersionsToDelete

public static List<Version> computeVersionsToDelete(List<Version> versions,
 int currentVersion,
 int clusterNumVersionsToPreserve,
 int storeNumVersionsToPreserve,
 boolean isMigrating)

Same version-deletion policy as retrieveVersionsToDelete(int), but over raw fields so callers that only hold a version snapshot (e.g. a child StoreInfo) can reuse it without a full Store. storeNumVersionsToPreserve is the store-level override (Store.NUM_VERSION_PRESERVE_NOT_SET when unset, in which case clusterNumVersionsToPreserve applies). Assumes versions is sorted by version number ascending.

isSystemStore

public boolean isSystemStore()

Specified by:

isSystemStore in interface Store

fixMissingFields

public void fixMissingFields()

Specified by:

fixMissingFields in interface Store

updateVersionForDaVinciHeartbeat

public void updateVersionForDaVinciHeartbeat(int versionNumber,
 boolean reported)

Specified by:

updateVersionForDaVinciHeartbeat in interface Store