Package com.linkedin.davinci.store.rocksdb

Class RocksDBStoragePartition

java.lang.Object
com.linkedin.davinci.store.AbstractStoragePartition
com.linkedin.davinci.store.rocksdb.RocksDBStoragePartition
Direct Known Subclasses:
ReplicationMetadataRocksDBStoragePartition
@NotThreadSafe public class RocksDBStoragePartition extends AbstractStoragePartition
In RocksDBStoragePartition, it assumes the update(insert/delete) will happen sequentially. If the batch push is bytewise-sorted by key, this class is leveraging SstFileWriter to generate the SST file directly and ingest all the generated SST files into the RocksDB database at the end of the push. If the ingestion is unsorted, this class is using the regular RocksDB interface to support update operations.

  • Field Details

    • ROCKSDB_ERROR_MESSAGE_FOR_RUNNING_OUT_OF_DISK_QUOTA

      protected static final String ROCKSDB_ERROR_MESSAGE_FOR_RUNNING_OUT_OF_DISK_QUOTA
      See Also:

    • READ_OPTIONS_DEFAULT

      protected static final org.rocksdb.ReadOptions READ_OPTIONS_DEFAULT

    • writeOptions

      protected final org.rocksdb.WriteOptions writeOptions
      Here RocksDB disables WAL, but relies on the 'flush', which will be invoked through sync() to avoid data loss during recovery.

    • iteratorReadOptions

      protected final org.rocksdb.ReadOptions iteratorReadOptions

    • replicaId

      protected final String replicaId

    • storeName

      protected final String storeName

    • storeNameAndVersion

      protected final String storeNameAndVersion

    • storeVersion

      protected final int storeVersion

    • partitionId

      protected final int partitionId

    • readCloseRWLock

      protected final ReentrantReadWriteLock readCloseRWLock
      Since all the modification functions are synchronized, we don't need any other synchronization for the update path to guard RocksDB closing behavior. The following readCloseRWLock is only used to guard get(byte[]) since we don't want to synchronize get requests.

    • rocksDB

      protected org.rocksdb.RocksDB rocksDB

    • deferredWrite

      protected final boolean deferredWrite
      Whether the input is sorted or not.
      deferredWrite = sortedInput => ingested via batch push which is sorted in VPJ, can use RocksDBSstFileWriter to ingest the input data to RocksDB
      !deferredWrite = !sortedInput => can not use RocksDBSstFileWriter for ingestion

    • readOnly

      protected final boolean readOnly
      Whether the database is read only or not.

    • writeOnly

      protected final boolean writeOnly

    • blobTransferInProgress

      protected final boolean blobTransferInProgress

    • readWriteLeaderForDefaultCF

      protected final boolean readWriteLeaderForDefaultCF

    • readWriteLeaderForRMDCF

      protected final boolean readWriteLeaderForRMDCF

    • columnFamilyHandleList

      protected final List<org.rocksdb.ColumnFamilyHandle> columnFamilyHandleList
      Column Family is the concept in RocksDB to create isolation between different value for the same key. All KVs are stored in `DEFAULT` column family, if no column family is specified. If we stores replication metadata in the RocksDB, we stored it in a separated column family. We will insert all the column family descriptors into columnFamilyDescriptors and pass it to RocksDB when opening the store, and it will fill the columnFamilyHandles with handles which will be used when we want to put/get/delete from different RocksDB column families.

    • columnFamilyDescriptors

      protected final List<org.rocksdb.ColumnFamilyDescriptor> columnFamilyDescriptors

  • Constructor Details

  • Method Details