Package com.linkedin.davinci.store.rocksdb

Class RocksDBStorageEngine

java.lang.Object

com.linkedin.davinci.store.AbstractStorageEngine<RocksDBStoragePartition>

com.linkedin.davinci.store.rocksdb.RocksDBStorageEngine

All Implemented Interfaces:

StorageEngine<RocksDBStoragePartition>, Closeable, AutoCloseable

public class RocksDBStorageEngine
extends AbstractStorageEngine<RocksDBStoragePartition>

Field Summary

Fields

Modifier and Type	Field	Description
static final String	SERVER_CONFIG_FILE_NAME

Fields inherited from class com.linkedin.davinci.store.AbstractStorageEngine

METADATA_PARTITION_ID

Constructor Summary

Constructors

Constructor

Description

RocksDBStorageEngine(VeniceStoreVersionConfig storeConfig, RocksDBStorageEngineFactory factory, String rocksDbPath, RocksDBMemoryStats rocksDBMemoryStats, RocksDBThrottler rocksDbThrottler, RocksDBServerConfig rocksDBServerConfig, InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, boolean replicationMetadataEnabled)

Method Summary

Modifier and Type	Method	Description
RocksDBStoragePartition	createStoragePartition(StoragePartitionConfig storagePartitionConfig)
void	drop()	Drop the whole store
protected void	dropPartitionDirectory(int partitionId)	Remove any on-disk artifacts for a partition that failed to restore.
AbstractStorageIterator	getIterator(int partitionId)
protected AbstractStoragePartition	getMetadataPartition()
Set<Integer>	getPersistedPartitionIds()	Retrieves the IDs of persisted partitions for the store.
StorageEngineStats	getStats()
PersistenceType	getType()
void	setRocksDBServerConfig(RocksDBServerConfig rocksDBServerConfig)
protected boolean	shouldDropPartitionOnRestoreFailure(int partitionId, Throwable cause)	Only drop a partition for failures that are clearly local to its on-disk state.

Methods inherited from class com.linkedin.davinci.store.AbstractStorageEngine

addStoragePartition, addStoragePartitionIfAbsent, adjustStoragePartition, beginBatchWrite, checkDatabaseIntegrity, clearPartitionOffset, clearStoreVersionState, close, closeMetadataPartition, closePartition, containsPartition, delete, deleteGlobalRtDivMetadata, deleteWithReplicationMetadata, dropPartition, dropPartition, endBatchWrite, executeWithSafeGuard, get, get, get, getByKeyPrefix, getGlobalRtDivMetadata, getNumberOfPartitions, getPartitionIds, getPartitionOffset, getPartitionOrThrow, getPartitions, getReplicationMetadata, getRWLockForPartitionOrThrow, getStoreVersionName, getStoreVersionState, isClosed, keyExists, put, put, putGlobalRtDivMetadata, putPartitionOffset, putReplicationMetadata, putStoreVersionState, putWithReplicationMetadata, reopenStoragePartition, restoreStoragePartitions, restoreStoragePartitions, restoreStoragePartitions, suppressLogs, sync, toString, updateStoreVersionStateCache

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

SERVER_CONFIG_FILE_NAME

public static final String SERVER_CONFIG_FILE_NAME

See Also:

• Constant Field Values

Constructor Details

RocksDBStorageEngine

public RocksDBStorageEngine(VeniceStoreVersionConfig storeConfig,
 RocksDBStorageEngineFactory factory,
 String rocksDbPath,
 RocksDBMemoryStats rocksDBMemoryStats,
 RocksDBThrottler rocksDbThrottler,
 RocksDBServerConfig rocksDBServerConfig,
 InternalAvroSpecificSerializer<StoreVersionState> storeVersionStateSerializer,
 InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer,
 boolean replicationMetadataEnabled)

Method Details

getMetadataPartition

protected AbstractStoragePartition getMetadataPartition()

Overrides:

getMetadataPartition in class AbstractStorageEngine<RocksDBStoragePartition>

getType

public PersistenceType getType()

getPersistedPartitionIds

public Set<Integer> getPersistedPartitionIds()

Retrieves the IDs of persisted partitions for the store. This method scans the existing store directory to identify the partition IDs that are retained and need to be persisted. Note: For stores with blob transfer enabled, temporary partition directories may exist if the instance previously fails during a transfer. In such cases, temporary directories should be excluded from the returned partition IDs.

Returns:

A set of IDs representing the persisted partitions.

createStoragePartition

public RocksDBStoragePartition createStoragePartition(StoragePartitionConfig storagePartitionConfig)

Specified by:

createStoragePartition in class AbstractStorageEngine<RocksDBStoragePartition>

dropPartitionDirectory

protected void dropPartitionDirectory(int partitionId)

Description copied from class: AbstractStorageEngine

Remove any on-disk artifacts for a partition that failed to restore. Subclasses backed by a real filesystem (e.g. RocksDB) should delete the partition directory so the partition can be re-bootstrapped from scratch. The default implementation is a no-op for in-memory engines.

Overrides:

dropPartitionDirectory in class AbstractStorageEngine<RocksDBStoragePartition>

shouldDropPartitionOnRestoreFailure

protected boolean shouldDropPartitionOnRestoreFailure(int partitionId,
 Throwable cause)

Only drop a partition for failures that are clearly local to its on-disk state. Environmental failures (disk full, permission denied, lock contention) must not trigger a drop — re-bootstrapping would amplify the problem or mask infra issues that need human attention. The drop-eligible cases are:

• Status.Code.Corruption: RocksDB has detected on-disk state it cannot read.
• Status.Code.IOError whose status state or exception message contains "No such file or directory": a partition file went missing. rocksdbjni 8.x does not expose a dedicated SubCode for path-not-found, so we fall back to a substring match on the human-readable status text. The substring match is gated by Code.IOError so we never drop on message text alone. Other IOError flavors (NoSpace, generic EIO, permission denied) are intentionally excluded because their status messages do not contain this substring.

Overrides:

shouldDropPartitionOnRestoreFailure in class AbstractStorageEngine<RocksDBStoragePartition>

drop

public void drop()

Description copied from class: AbstractStorageEngine

Drop the whole store

Specified by:

drop in interface StorageEngine<RocksDBStoragePartition>

Overrides:

drop in class AbstractStorageEngine<RocksDBStoragePartition>

setRocksDBServerConfig

public void setRocksDBServerConfig(RocksDBServerConfig rocksDBServerConfig)

getIterator

public AbstractStorageIterator getIterator(int partitionId)

Specified by:

getIterator in interface StorageEngine<RocksDBStoragePartition>

Overrides:

getIterator in class AbstractStorageEngine<RocksDBStoragePartition>

getStats

public StorageEngineStats getStats()