Package com.linkedin.davinci.store

Class AbstractStoragePartition

java.lang.Object
com.linkedin.davinci.store.AbstractStoragePartition
Direct Known Subclasses:
BlackHoleStorageEnginePartition, InMemoryStoragePartition, RocksDBStoragePartition, VeniceStoreCacheStoragePartition
public abstract class AbstractStoragePartition extends Object
An abstract implementation of a storage partition. This could be a database in BDB environment or a concurrent hashmap in the case of an in-memory implementation depending on the storage-partition model.

  • Field Details

    • partitionId

      protected final Integer partitionId

  • Constructor Details

    • AbstractStoragePartition

      public AbstractStoragePartition(Integer partitionId)

  • Method Details

    • getPartitionId

      public Integer getPartitionId()
      returns the id of this partition

    • put

      public abstract void put(byte[] key, byte[] value)
      Puts a value into the partition database

    • put

      public abstract void put(byte[] key, ByteBuffer value)

    • put

      public abstract <K, V> void put(K key, V value)

    • get

      public abstract byte[] get(byte[] key)
      Get a value from the partition database
      Parameters:
      key - key to be retrieved
      Returns:
      null if the key does not exist, byte[] value if it exists.

    • get

      public ByteBuffer get(byte[] key, ByteBuffer valueToBePopulated)

    • get

      public abstract <K, V> V get(K key)
      Get a Value from the partition database
      Type Parameters:
      K - the type for Key
      V - the type for the return value
      Parameters:
      key - key to be retrieved
      Returns:
      null if the key does not exist, V value if it exists

    • get

      public abstract byte[] get(ByteBuffer key)

    • getByKeyPrefix

      public abstract void getByKeyPrefix(byte[] keyPrefix, BytesStreamingCallback callback)
      Populate provided callback with key-value pairs from the partition database where the keys have provided prefix. If prefix is null, callback will be populated will all key-value pairs from the partition database.
      Parameters:
      keyPrefix -
      callback -

    • delete

      public abstract void delete(byte[] key)
      Delete a key from the partition database

    • sync

      public abstract Map<String,String> sync()
      Sync current database.
      Returns:
      Database related info, which is required to be checkpointed.

    • drop

      public abstract void drop()
      Drop when it is not required anymore.

    • close

      public abstract void close()
      Close the specific partition

    • reopen

      public void reopen()
      Reopen the database.

    • verifyConfig

      public abstract boolean verifyConfig(StoragePartitionConfig storagePartitionConfig)
      Check whether current storage partition verifyConfig the given partition config
      Parameters:
      storagePartitionConfig -
      Returns:

    • createSnapshot

      public abstract void createSnapshot()
      Creates a snapshot of the current state of the storage if the blob transfer feature is enabled via the store configuration

    • checkDatabaseIntegrity

      public boolean checkDatabaseIntegrity(Map<String,String> checkpointedInfo)
      checks whether the current state of the database is valid during the start of ingestion.

    • beginBatchWrite

      public void beginBatchWrite(Map<String,String> checkpointedInfo, Optional<Supplier<byte[]>> checksumSupplier)

    • endBatchWrite

      public void endBatchWrite()

    • getPartitionSizeInBytes

      public abstract long getPartitionSizeInBytes()
      Get the partition database size in bytes
      Returns:
      partition database size

    • validateBatchIngestion

      public boolean validateBatchIngestion()

    • putWithReplicationMetadata

      public void putWithReplicationMetadata(byte[] key, ByteBuffer value, byte[] metadata)
      This API takes in value and metadata as ByteBuffer format and put it into RocksDB. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will UnsupportedOperationException.

    • putWithReplicationMetadata

      public void putWithReplicationMetadata(byte[] key, byte[] value, byte[] metadata)
      This API takes in value and metadata as byte array format and put it into RocksDB. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will VeniceUnsupportedOperationException.

    • putReplicationMetadata

      public void putReplicationMetadata(byte[] key, byte[] metadata)

    • getReplicationMetadata

      public byte[] getReplicationMetadata(ByteBuffer key)
      This API retrieves replication metadata from replicationMetadataColumnFamily. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will VeniceUnsupportedOperationException.

    • deleteWithReplicationMetadata

      public void deleteWithReplicationMetadata(byte[] key, byte[] metadata)
      This API deletes a record from RocksDB but updates the metadata in ByteBuffer format and puts it into RocksDB. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will VeniceUnsupportedOperationException.

    • getRmdByteUsage

      public long getRmdByteUsage()

    • getIterator

      public AbstractStorageIterator getIterator()