Package com.linkedin.davinci.kafka.consumer

Class AbstractStoreBufferService

java.lang.Object

com.linkedin.venice.service.AbstractVeniceService

com.linkedin.davinci.kafka.consumer.AbstractStoreBufferService

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

SeparatedStoreBufferService, StoreBufferService

public abstract class AbstractStoreBufferService
extends AbstractVeniceService

Abstract class capturing the responsibilities of drainers threads doing store ingestion.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService

AbstractVeniceService.ServiceState

Field Summary

Fields inherited from class com.linkedin.venice.service.AbstractVeniceService

logger, serviceState

Constructor Summary

Constructors

Constructor	Description
AbstractStoreBufferService()

Method Summary

Modifier and Type	Method	Description
abstract void	drainBufferedRecordsFromTopicPartition(PubSubTopicPartition topicPartition)	This method will wait for all the messages to be processed (persisted to disk) that are already queued up to drainer till now.
abstract CompletableFuture<Void>	execSyncOffsetCommandAsync(PubSubTopicPartition topicPartition, StoreIngestionTask ingestionTask)
abstract long	getMaxMemoryUsagePerDrainer()
abstract long	getMinMemoryUsagePerDrainer()
abstract long	getTotalMemoryUsage()
abstract long	getTotalRemainingMemory()
abstract void	putConsumerRecord(PubSubMessage<KafkaKey,KafkaMessageEnvelope,Long> consumerRecord, StoreIngestionTask ingestionTask, LeaderProducedRecordContext leaderProducedRecordContext, int partition, String kafkaUrl, long beforeProcessingRecordTimestampNs)

Methods inherited from class com.linkedin.venice.service.AbstractVeniceService

close, getName, isRunning, start, startInner, stop, stopInner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

AbstractStoreBufferService

public AbstractStoreBufferService()

Method Details

putConsumerRecord

public abstract void putConsumerRecord(PubSubMessage<KafkaKey,KafkaMessageEnvelope,Long> consumerRecord,
 StoreIngestionTask ingestionTask,
 LeaderProducedRecordContext leaderProducedRecordContext,
 int partition,
 String kafkaUrl,
 long beforeProcessingRecordTimestampNs)
                                throws InterruptedException

Throws:

InterruptedException

drainBufferedRecordsFromTopicPartition

public abstract void drainBufferedRecordsFromTopicPartition(PubSubTopicPartition topicPartition)
                                                     throws InterruptedException

This method will wait for all the messages to be processed (persisted to disk) that are already queued up to drainer till now.

Throws:

InterruptedException

getTotalMemoryUsage

public abstract long getTotalMemoryUsage()

getTotalRemainingMemory

public abstract long getTotalRemainingMemory()

getMaxMemoryUsagePerDrainer

public abstract long getMaxMemoryUsagePerDrainer()

getMinMemoryUsagePerDrainer

public abstract long getMinMemoryUsagePerDrainer()

execSyncOffsetCommandAsync

public abstract CompletableFuture<Void> execSyncOffsetCommandAsync(PubSubTopicPartition topicPartition,
 StoreIngestionTask ingestionTask)
                                                            throws InterruptedException

Throws:

InterruptedException