Package com.linkedin.davinci.storage

Class StorageEngineMetadataService

java.lang.Object

com.linkedin.venice.service.AbstractVeniceService

com.linkedin.davinci.storage.StorageEngineMetadataService

All Implemented Interfaces:

StorageMetadataService, OffsetManager, Closeable, AutoCloseable

public class StorageEngineMetadataService
extends AbstractVeniceService
implements StorageMetadataService

StorageEngineMetadataService is wrapper service on top of storageEngineRepository to serve read/write to storage metadata. It contains methods to read/write/clear the store version state and partition offset that are stored in metadata partition.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService

AbstractVeniceService.ServiceState

Field Summary

Fields inherited from class com.linkedin.venice.service.AbstractVeniceService

logger, serviceState

Constructor Summary

Constructors

Constructor	Description
StorageEngineMetadataService(StorageEngineRepository storageEngineRepository, InternalAvroSpecificSerializer<PartitionState> serializer)

Method Summary

Modifier and Type	Method	Description
void	clearOffset(String topicName, int partitionId)
void	clearStoreVersionState(String topicName)	This will clear all metadata, including store-version state and partition states, tied to .
StoreVersionState	computeStoreVersionState(String topicName, Function<StoreVersionState,StoreVersionState> mapFunction)
OffsetRecord	getLastOffset(String topicName, int partitionId, PubSubContext pubSubContext)	Gets the Last Known persisted offset of this consumer.
StoreVersionState	getStoreVersionState(String topicName)	Gets the currently-persisted StoreVersionState for this topic.
void	put(String topicName, int partitionId, OffsetRecord record)	Records the offset with underlying/external storage.
boolean	startInner()
void	stopInner()

Methods inherited from class com.linkedin.venice.service.AbstractVeniceService

close, getName, isRunning, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.davinci.storage.StorageMetadataService

getStoreVersionCompressionDictionary

Constructor Details

StorageEngineMetadataService

public StorageEngineMetadataService(StorageEngineRepository storageEngineRepository,
 InternalAvroSpecificSerializer<PartitionState> serializer)

Method Details

put

public void put(String topicName,
 int partitionId,
 OffsetRecord record)
         throws VeniceException

Description copied from interface: OffsetManager

Records the offset with underlying/external storage. Persistence to disk happens in configurable time interval by a background thread. For example in case of BDB the check pointer thread can be configured to do this.

Specified by:

put in interface OffsetManager

Parameters:

topicName - kafka topic to which the consumer thread is registered to.

partitionId - kafka partition id for which the consumer thread is registered to.

record - OffSetRecord containing last read offset for the topic and partition combination.

Throws:

VeniceException

clearOffset

public void clearOffset(String topicName,
 int partitionId)

Specified by:

clearOffset in interface OffsetManager

getLastOffset

public OffsetRecord getLastOffset(String topicName,
 int partitionId,
 PubSubContext pubSubContext)
                           throws VeniceException

Description copied from interface: OffsetManager

Gets the Last Known persisted offset of this consumer.

Specified by:

getLastOffset in interface OffsetManager

Parameters:

topicName - kafka topic to which the consumer thread is registered to.

partitionId - kafka partition id for which the consumer thread is registered to.

pubSubContext - PubSubContext for passing PubSub related dependencies

Returns:

OffsetRecord - contains offset and time when it was recorded before the consumer thread went down. consumer

Throws:

VeniceException

startInner

public boolean startInner()
                   throws Exception

Specified by:

startInner in class AbstractVeniceService

Returns:

true if the service is completely started, false if it is still starting asynchronously (in this case, it is the implementer's responsibility to set AbstractVeniceService.serviceState to AbstractVeniceService.ServiceState.STARTED upon completion of the async work).

Throws:

Exception

stopInner

public void stopInner()
               throws Exception

Specified by:

stopInner in class AbstractVeniceService

Throws:

Exception

computeStoreVersionState

public StoreVersionState computeStoreVersionState(String topicName,
 Function<StoreVersionState,StoreVersionState> mapFunction)
                                           throws VeniceException

Specified by:

computeStoreVersionState in interface StorageMetadataService

Throws:

VeniceException

clearStoreVersionState

public void clearStoreVersionState(String topicName)

Description copied from interface: StorageMetadataService

This will clear all metadata, including store-version state and partition states, tied to .

Specified by:

clearStoreVersionState in interface StorageMetadataService

Parameters:

topicName - to be cleared

getStoreVersionState

public StoreVersionState getStoreVersionState(String topicName)
                                       throws VeniceException

Description copied from interface: StorageMetadataService

Gets the currently-persisted StoreVersionState for this topic.

Specified by:

getStoreVersionState in interface StorageMetadataService

Parameters:

topicName - kafka topic to which the consumer thread is registered to.

Returns:

an instance of StoreVersionState corresponding to this topic, or null if there isn't any.

Throws:

VeniceException