Package com.linkedin.davinci.storage
Class StorageService
java.lang.Object
com.linkedin.venice.service.AbstractVeniceService
com.linkedin.davinci.storage.StorageService
- All Implemented Interfaces:
Closeable
,AutoCloseable
Storage interface to Venice Server, Da Vinci and Isolated Ingestion Service. Manages creation and deletion of storage
engines and partitions.
Use StorageEngineRepository, if read only access is desired for the Storage Engines.
-
Nested Class Summary
Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService
AbstractVeniceService.ServiceState
-
Field Summary
Fields inherited from class com.linkedin.venice.service.AbstractVeniceService
logger, serviceState
-
Constructor Summary
ConstructorsConstructorDescriptionStorageService
(VeniceConfigLoader configLoader, AggVersionedStorageEngineStats storageEngineStats, RocksDBMemoryStats rocksDBMemoryStats, InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, ReadOnlyStoreRepository storeRepository) StorageService
(VeniceConfigLoader configLoader, AggVersionedStorageEngineStats storageEngineStats, RocksDBMemoryStats rocksDBMemoryStats, InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, ReadOnlyStoreRepository storeRepository, boolean restoreDataPartitions, boolean restoreMetadataPartitions) StorageService
(VeniceConfigLoader configLoader, AggVersionedStorageEngineStats storageEngineStats, RocksDBMemoryStats rocksDBMemoryStats, InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, ReadOnlyStoreRepository storeRepository, boolean restoreDataPartitions, boolean restoreMetadataPartitions, Function<String, Boolean> checkWhetherStorageEngineShouldBeKeptOrNot) -
Method Summary
Modifier and TypeMethodDescriptionvoid
void
cleanupAllStores
(VeniceConfigLoader configLoader) void
void
closeStorageEngine
(String kafkaTopic) void
closeStorePartition
(VeniceStoreVersionConfig storeConfig, int partition) void
dropStorePartition
(VeniceStoreVersionConfig storeConfig, int partition) Drops the partition of the specified store version in the storage service.void
dropStorePartition
(VeniceStoreVersionConfig storeConfig, int partition, boolean removeEmptyStorageEngine) Drops the partition of the specified store version in the storage service.void
forceStorageEngineCleanup
(String kafkaTopic) This function is used to forcely clean up all the databases belonging to .getInternalStorageEngineFactory
(VeniceStoreVersionConfig storeConfig) This method should ideally be Private, but marked as public for validating the result.Optional<org.rocksdb.Statistics>
getStorageEngine
(String kafkaTopic) getUserPartitions
(String kafkaTopicName) openStore
(VeniceStoreVersionConfig storeConfig, Supplier<StoreVersionState> initialStoreVersionStateSupplier) Creates a StorageEngineFactory for the persistence type if not already present.openStoreForNewPartition
(VeniceStoreVersionConfig storeConfig, int partitionId, Supplier<StoreVersionState> initialStoreVersionStateSupplier) void
removeStorageEngine
(String kafkaTopic) boolean
void
-
Constructor Details
-
StorageService
public StorageService(VeniceConfigLoader configLoader, AggVersionedStorageEngineStats storageEngineStats, RocksDBMemoryStats rocksDBMemoryStats, InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, ReadOnlyStoreRepository storeRepository, boolean restoreDataPartitions, boolean restoreMetadataPartitions, Function<String, Boolean> checkWhetherStorageEngineShouldBeKeptOrNot) -
StorageService
public StorageService(VeniceConfigLoader configLoader, AggVersionedStorageEngineStats storageEngineStats, RocksDBMemoryStats rocksDBMemoryStats, InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, ReadOnlyStoreRepository storeRepository, boolean restoreDataPartitions, boolean restoreMetadataPartitions) -
StorageService
public StorageService(VeniceConfigLoader configLoader, AggVersionedStorageEngineStats storageEngineStats, RocksDBMemoryStats rocksDBMemoryStats, InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, ReadOnlyStoreRepository storeRepository)
-
-
Method Details
-
openStoreForNewPartition
public AbstractStorageEngine openStoreForNewPartition(VeniceStoreVersionConfig storeConfig, int partitionId, Supplier<StoreVersionState> initialStoreVersionStateSupplier) -
getStoreVersionStateSyncer
-
getInternalStorageEngineFactory
This method should ideally be Private, but marked as public for validating the result.- Parameters:
storeConfig
- StoreConfig of the store.- Returns:
- Factory corresponding to the store.
-
getRocksDBAggregatedStatistics
-
openStore
public AbstractStorageEngine openStore(VeniceStoreVersionConfig storeConfig, Supplier<StoreVersionState> initialStoreVersionStateSupplier) Creates a StorageEngineFactory for the persistence type if not already present. Creates a new storage engine for the given store in the factory and registers the storage engine with the store repository.- Parameters:
storeConfig
- The store specific propertiesinitialStoreVersionStateSupplier
- invoked to initialize the SVS when a brand-new storage engine is created- Returns:
- StorageEngine that was created for the given store definition.
-
checkWhetherStoragePartitionsShouldBeKeptOrNot
-
dropStorePartition
Drops the partition of the specified store version in the storage service. When all data partitions are dropped, it will also drop the storage engine of the specific store version.- Parameters:
storeConfig
- config of the store version.partition
- partition ID to be dropped.
-
dropStorePartition
public void dropStorePartition(VeniceStoreVersionConfig storeConfig, int partition, boolean removeEmptyStorageEngine) Drops the partition of the specified store version in the storage service.- Parameters:
storeConfig
- config of the store version.partition
- partition ID to be dropped.removeEmptyStorageEngine
- Whether to delete the storage engine when there is no remaining data partition.
-
closeStorePartition
-
removeStorageEngine
-
forceStorageEngineCleanup
This function is used to forcely clean up all the databases belonging to . This function will only be used when theremoveStorageEngine(String)
function can't handle some edge case, such as some partitions are lingering, which are not visible to the correspondingAbstractStorageEngine
-
closeStorageEngine
-
cleanupAllStores
-
getUserPartitions
-
closeAllStorageEngines
public void closeAllStorageEngines() -
getStorageEngineRepository
-
getStorageEngine
-
getStoreAndUserPartitionsMapping
-
startInner
- Specified by:
startInner
in classAbstractVeniceService
- Returns:
- true if the service is completely started,
false if it is still starting asynchronously (in this case, it is the implementer's
responsibility to set
AbstractVeniceService.serviceState
toAbstractVeniceService.ServiceState.STARTED
upon completion of the async work). - Throws:
Exception
-
stopInner
- Specified by:
stopInner
in classAbstractVeniceService
- Throws:
VeniceException
-