Package com.linkedin.davinci.store

Class AbstractStoragePartition

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected java.lang.Integer partitionId  

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      void beginBatchWrite​(java.util.Map<java.lang.String,​java.lang.String> checkpointedInfo, java.util.Optional<java.util.function.Supplier<byte[]>> checksumSupplier)  
      boolean checkDatabaseIntegrity​(java.util.Map<java.lang.String,​java.lang.String> checkpointedInfo)
      checks whether the current state of the database is valid during the start of ingestion.
      abstract void close()
      Close the specific partition
      abstract void createSnapshot()
      Creates a snapshot of the current state of the storage if the blob transfer feature is enabled via the store configuration
      abstract void delete​(byte[] key)
      Delete a key from the partition database
      void deleteWithReplicationMetadata​(byte[] key, byte[] metadata)
      This API deletes a record from RocksDB but updates the metadata in ByteBuffer format and puts it into RocksDB.
      abstract void drop()
      Drop when it is not required anymore.
      void endBatchWrite()  
      abstract byte[] get​(byte[] key)
      Get a value from the partition database
      java.nio.ByteBuffer get​(byte[] key, java.nio.ByteBuffer valueToBePopulated)  
      abstract byte[] get​(java.nio.ByteBuffer key)  
      abstract <K,​V>
      V      		 get​(K key)
      Get a Value from the partition database
      abstract void getByKeyPrefix​(byte[] keyPrefix, BytesStreamingCallback callback)
      Populate provided callback with key-value pairs from the partition database where the keys have provided prefix.
      AbstractStorageIterator getIterator()  
      java.lang.Integer getPartitionId()
      returns the id of this partition
      abstract long getPartitionSizeInBytes()
      Get the partition database size in bytes
      byte[] getReplicationMetadata​(java.nio.ByteBuffer key)
      This API retrieves replication metadata from replicationMetadataColumnFamily.
      long getRmdByteUsage()  
      abstract void put​(byte[] key, byte[] value)
      Puts a value into the partition database
      abstract void put​(byte[] key, java.nio.ByteBuffer value)  
      abstract <K,​V>
      void      		 put​(K key, V value)  
      void putReplicationMetadata​(byte[] key, byte[] metadata)  
      void putWithReplicationMetadata​(byte[] key, byte[] value, byte[] metadata)
      This API takes in value and metadata as byte array format and put it into RocksDB.
      void putWithReplicationMetadata​(byte[] key, java.nio.ByteBuffer value, byte[] metadata)
      This API takes in value and metadata as ByteBuffer format and put it into RocksDB.
      void reopen()
      Reopen the database.
      abstract java.util.Map<java.lang.String,​java.lang.String> sync()
      Sync current database.
      boolean validateBatchIngestion()  
      abstract boolean verifyConfig​(StoragePartitionConfig storagePartitionConfig)
      Check whether current storage partition verifyConfig the given partition config

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • partitionId

        protected final java.lang.Integer partitionId

    • Constructor Detail

      • AbstractStoragePartition

        public AbstractStoragePartition​(java.lang.Integer partitionId)

    • Method Detail

      • getPartitionId

        public java.lang.Integer getPartitionId()
        returns the id of this partition

      • put

        public abstract void put​(byte[] key,
                         byte[] value)
        Puts a value into the partition database

      • put

        public abstract void put​(byte[] key,
                         java.nio.ByteBuffer value)

      • put

        public abstract <K,​V> void put​(K key,
                                     V value)

      • get

        public abstract byte[] get​(byte[] key)
        Get a value from the partition database
        Parameters:
        key - key to be retrieved
        Returns:
        null if the key does not exist, byte[] value if it exists.

      • get

        public java.nio.ByteBuffer get​(byte[] key,
                               java.nio.ByteBuffer valueToBePopulated)

      • get

        public abstract <K,​V> V get​(K key)
        Get a Value from the partition database
        Type Parameters:
        K - the type for Key
        V - the type for the return value
        Parameters:
        key - key to be retrieved
        Returns:
        null if the key does not exist, V value if it exists

      • get

        public abstract byte[] get​(java.nio.ByteBuffer key)

      • getByKeyPrefix

        public abstract void getByKeyPrefix​(byte[] keyPrefix,
                                    BytesStreamingCallback callback)
        Populate provided callback with key-value pairs from the partition database where the keys have provided prefix. If prefix is null, callback will be populated will all key-value pairs from the partition database.
        Parameters:
        keyPrefix -
        callback -

      • delete

        public abstract void delete​(byte[] key)
        Delete a key from the partition database

      • sync

        public abstract java.util.Map<java.lang.String,​java.lang.String> sync()
        Sync current database.
        Returns:
        Database related info, which is required to be checkpointed.

      • drop

        public abstract void drop()
        Drop when it is not required anymore.

      • close

        public abstract void close()
        Close the specific partition

      • reopen

        public void reopen()
        Reopen the database.

      • verifyConfig

        public abstract boolean verifyConfig​(StoragePartitionConfig storagePartitionConfig)
        Check whether current storage partition verifyConfig the given partition config
        Parameters:
        storagePartitionConfig -
        Returns:

      • createSnapshot

        public abstract void createSnapshot()
        Creates a snapshot of the current state of the storage if the blob transfer feature is enabled via the store configuration

      • checkDatabaseIntegrity

        public boolean checkDatabaseIntegrity​(java.util.Map<java.lang.String,​java.lang.String> checkpointedInfo)
        checks whether the current state of the database is valid during the start of ingestion.

      • beginBatchWrite

        public void beginBatchWrite​(java.util.Map<java.lang.String,​java.lang.String> checkpointedInfo,
                            java.util.Optional<java.util.function.Supplier<byte[]>> checksumSupplier)

      • endBatchWrite

        public void endBatchWrite()

      • getPartitionSizeInBytes

        public abstract long getPartitionSizeInBytes()
        Get the partition database size in bytes
        Returns:
        partition database size

      • validateBatchIngestion

        public boolean validateBatchIngestion()

      • putWithReplicationMetadata

        public void putWithReplicationMetadata​(byte[] key,
                                       java.nio.ByteBuffer value,
                                       byte[] metadata)
        This API takes in value and metadata as ByteBuffer format and put it into RocksDB. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will UnsupportedOperationException.

      • putWithReplicationMetadata

        public void putWithReplicationMetadata​(byte[] key,
                                       byte[] value,
                                       byte[] metadata)
        This API takes in value and metadata as byte array format and put it into RocksDB. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will VeniceUnsupportedOperationException.

      • putReplicationMetadata

        public void putReplicationMetadata​(byte[] key,
                                   byte[] metadata)

      • getReplicationMetadata

        public byte[] getReplicationMetadata​(java.nio.ByteBuffer key)
        This API retrieves replication metadata from replicationMetadataColumnFamily. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will VeniceUnsupportedOperationException.

      • deleteWithReplicationMetadata

        public void deleteWithReplicationMetadata​(byte[] key,
                                          byte[] metadata)
        This API deletes a record from RocksDB but updates the metadata in ByteBuffer format and puts it into RocksDB. Only ReplicationMetadataRocksDBStoragePartition will execute this method, other storage partition implementation will VeniceUnsupportedOperationException.

      • getRmdByteUsage

        public long getRmdByteUsage()