Package com.linkedin.davinci.store

Interface StorageEngine<Partition extends AbstractStoragePartition>

All Superinterfaces:
AutoCloseable, Closeable
All Known Implementing Classes:
AbstractStorageEngine, BlackHoleStorageEngine, DelegatingStorageEngine, InMemoryStorageEngine, RocksDBStorageEngine, VeniceStoreCacheStorageEngine
public interface StorageEngine<Partition extends AbstractStoragePartition> extends Closeable

  • Method Details

    • getStoreVersionName

      String getStoreVersionName()

    • getType

      PersistenceType getType()

    • getPersistedPartitionIds

      Set<Integer> getPersistedPartitionIds()

    • adjustStoragePartition

      void adjustStoragePartition(int partitionId, StoragePartitionAdjustmentTrigger mode, StoragePartitionConfig partitionConfig)
      Adjust the opened storage partition according to the provided storagePartitionConfig. It will throw exception if there is no opened storage partition for the given partition id. The reason to have is mainly used to ease the unit test.

    • addStoragePartitionIfAbsent

      void addStoragePartitionIfAbsent(int partitionId)

    • addStoragePartition

      void addStoragePartition(StoragePartitionConfig storagePartitionConfig)

    • closePartition

      void closePartition(int partitionId)

    • closeMetadataPartition

      void closeMetadataPartition()

    • dropPartition

      void dropPartition(int partitionId)
      Removes and returns a partition from the current store
      Parameters:
      partitionId - - id of partition to retrieve and remove

    • dropPartition

      void dropPartition(int partitionId, boolean dropMetadataPartitionWhenEmpty)
      Removes and returns a partition from the current store
      Parameters:
      partitionId - - id of partition to retrieve and remove
      dropMetadataPartitionWhenEmpty - - if true, the whole store will be dropped if ALL partitions are removed

    • drop

      void drop()
      Drop the whole store

    • sync

      Map<String,String> sync(int partitionId)

    • close

      void close() throws VeniceException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      VeniceException

    • isClosed

      boolean isClosed()

    • checkDatabaseIntegrity

      boolean checkDatabaseIntegrity(int partitionId, Map<String,String> checkpointedInfo, StoragePartitionConfig storagePartitionConfig)
      checks whether the current state of the database is valid during the start of ingestion.

    • beginBatchWrite

      void beginBatchWrite(StoragePartitionConfig storagePartitionConfig, Map<String,String> checkpointedInfo, Optional<Supplier<byte[]>> checksumSupplier)
      A lot of storage engines support efficient methods for performing large number of writes (puts/deletes) against the data source. This method puts the storage engine in this batch write mode

    • endBatchWrite

      void endBatchWrite(StoragePartitionConfig storagePartitionConfig)

    • reopenStoragePartition

      void reopenStoragePartition(int partitionId)
      Reopen the underlying database.

    • put

      void put(int partitionId, byte[] key, byte[] value) throws VeniceException
      Throws:
      VeniceException

    • put

      void put(int partitionId, byte[] key, ByteBuffer value) throws VeniceException
      Throws:
      VeniceException

    • putWithReplicationMetadata

      void putWithReplicationMetadata(int partitionId, byte[] key, ByteBuffer value, byte[] replicationMetadata) throws VeniceException
      Throws:
      VeniceException

    • putReplicationMetadata

      void putReplicationMetadata(int partitionId, byte[] key, byte[] replicationMetadata) throws VeniceException
      Throws:
      VeniceException

    • get

      byte[] get(int partitionId, byte[] key) throws VeniceException
      Throws:
      VeniceException

    • get

      ByteBuffer get(int partitionId, byte[] key, ByteBuffer valueToBePopulated) throws VeniceException
      Throws:
      VeniceException

    • get

      byte[] get(int partitionId, ByteBuffer keyBuffer) throws VeniceException
      Throws:
      VeniceException

    • getByKeyPrefix

      void getByKeyPrefix(int partitionId, byte[] partialKey, BytesStreamingCallback bytesStreamingCallback)

    • delete

      void delete(int partitionId, byte[] key) throws VeniceException
      Throws:
      VeniceException

    • deleteWithReplicationMetadata

      void deleteWithReplicationMetadata(int partitionId, byte[] key, byte[] replicationMetadata) throws VeniceException
      Throws:
      VeniceException

    • getReplicationMetadata

      byte[] getReplicationMetadata(int partitionId, ByteBuffer key)

    • putPartitionOffset

      void putPartitionOffset(int partitionId, OffsetRecord offsetRecord)
      Put the offset associated with the partitionId into the metadata partition.

    • getPartitionOffset

      Optional<OffsetRecord> getPartitionOffset(int partitionId, PubSubContext pubSubContext)
      Retrieve the offset associated with the partitionId from the metadata partition.

    • clearPartitionOffset

      void clearPartitionOffset(int partitionId)
      Clear the offset associated with the partitionId in the metadata partition.

    • putStoreVersionState

      void putStoreVersionState(StoreVersionState versionState)
      Put the store version state into the metadata partition.

    • updateStoreVersionStateCache

      void updateStoreVersionStateCache(StoreVersionState versionState)
      Used in ingestion isolation mode update the storage engine's cache in sync with the updates to the state in MainIngestionStorageMetadataService

    • getStoreVersionState

      StoreVersionState getStoreVersionState()
      Retrieve the store version state from the metadata partition.

    • clearStoreVersionState

      void clearStoreVersionState()
      Clear the store version state in the metadata partition.

    • containsPartition

      boolean containsPartition(int partitionId)
      Return true or false based on whether a given partition exists within this storage engine
      Parameters:
      partitionId - The partition to look for
      Returns:
      True/False, does the partition exist on this node

    • getPartitionIds

      Set<Integer> getPartitionIds()
      Get all Partition Ids which are assigned to the current Node.
      Returns:
      partition Ids that are hosted in the current Storage Engine.

    • getPartitionOrThrow

      Partition getPartitionOrThrow(int partitionId)

    • getIterator

      AbstractStorageIterator getIterator(int partitionId)

    • suppressLogs

      void suppressLogs(boolean b)

    • getStats

      default StorageEngineStats getStats()