Package com.linkedin.davinci.ingestion.main

Class MainIngestionMonitorService

java.lang.Object

com.linkedin.venice.service.AbstractVeniceService

com.linkedin.davinci.ingestion.main.MainIngestionMonitorService

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public class MainIngestionMonitorService
extends AbstractVeniceService

This class is the listener service in main process which handles various kinds of reports sent from isolated ingestion service. MainIngestionMonitorService itself is a Netty based server implementation, and the main report handling logics happens in MainIngestionReportHandler. Besides reports handling, it also maintains two executor services to send heartbeat check and collect metrics to/from child process. Also, it maintains status for all the ongoing/completed topic partition ingestion tasks, which helps IsolatedIngestionBackend to check which process a topic partition storage is located, as well as status recovery when child process crashed and restarted.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService

AbstractVeniceService.ServiceState

Field Summary

Fields inherited from class com.linkedin.venice.service.AbstractVeniceService

logger, serviceState

Constructor Summary

Constructors

Constructor	Description
MainIngestionMonitorService(IsolatedIngestionBackend ingestionBackend, VeniceConfigLoader configLoader)

Method Summary

Modifier and Type	Method	Description
void	addIngestionNotifier(VeniceNotifier ingestionListener)
void	addPushStatusNotifier(VeniceNotifier pushStatusNotifier)
void	cleanupTopicPartitionState(java.lang.String topicName, int partitionId)
void	cleanupTopicState(java.lang.String topicName)
java.util.concurrent.locks.ReentrantReadWriteLock	getForkProcessActionLock()
java.util.List<VeniceNotifier>	getIngestionNotifier()
java.util.List<VeniceNotifier>	getPushStatusNotifierList()
MainIngestionStorageMetadataService	getStorageMetadataService()
KafkaStoreIngestionService	getStoreIngestionService()
java.util.Map<java.lang.String,MainTopicIngestionStatus>	getTopicIngestionStatusMap()
long	getTopicPartitionCount(java.lang.String topicName)
MainPartitionIngestionStatus	getTopicPartitionIngestionStatus(java.lang.String topicName, int partitionId)
java.util.Map<java.lang.String,java.util.Map<java.lang.Integer,java.lang.Boolean>>	getTopicPartitionLeaderStatusMap()
boolean	isTopicPartitionInLeaderState(java.lang.String topicName, int partitionId)
void	setMetricsRepository(io.tehuti.metrics.MetricsRepository metricsRepository)
void	setStorageMetadataService(MainIngestionStorageMetadataService storageMetadataService)
void	setStoreIngestionService(KafkaStoreIngestionService storeIngestionService)
void	setStoreRepository(ReadOnlyStoreRepository storeRepository)
void	setTopicIngestionToFollowerState(java.lang.String topicName, int partitionId)
void	setTopicPartitionToLeaderState(java.lang.String topicName, int partitionId)
void	setVersionPartitionToIsolatedIngestion(java.lang.String topicName, int partitionId)
void	setVersionPartitionToLocalIngestion(java.lang.String topicName, int partitionId)
boolean	startInner()
void	stopInner()

Methods inherited from class com.linkedin.venice.service.AbstractVeniceService

close, getName, isRunning, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MainIngestionMonitorService

public MainIngestionMonitorService(IsolatedIngestionBackend ingestionBackend,
                                   VeniceConfigLoader configLoader)

Method Detail

startInner

public boolean startInner()
                   throws java.lang.Exception

Specified by:

startInner in class AbstractVeniceService

Returns:

true if the service is completely started, false if it is still starting asynchronously (in this case, it is the implementer's responsibility to set AbstractVeniceService.serviceState to AbstractVeniceService.ServiceState.STARTED upon completion of the async work).

Throws:

java.lang.Exception

stopInner

public void stopInner()
               throws java.lang.Exception

Specified by:

stopInner in class AbstractVeniceService

Throws:

java.lang.Exception

addIngestionNotifier

public void addIngestionNotifier(VeniceNotifier ingestionListener)

getIngestionNotifier

public java.util.List<VeniceNotifier> getIngestionNotifier()

addPushStatusNotifier

public void addPushStatusNotifier(VeniceNotifier pushStatusNotifier)

getPushStatusNotifierList

public java.util.List<VeniceNotifier> getPushStatusNotifierList()

setMetricsRepository

public void setMetricsRepository(io.tehuti.metrics.MetricsRepository metricsRepository)

setStorageMetadataService

public void setStorageMetadataService(MainIngestionStorageMetadataService storageMetadataService)

getStorageMetadataService

public MainIngestionStorageMetadataService getStorageMetadataService()

setStoreIngestionService

public void setStoreIngestionService(KafkaStoreIngestionService storeIngestionService)

getStoreIngestionService

public KafkaStoreIngestionService getStoreIngestionService()

setStoreRepository

public void setStoreRepository(ReadOnlyStoreRepository storeRepository)

isTopicPartitionInLeaderState

public boolean isTopicPartitionInLeaderState(java.lang.String topicName,
                                             int partitionId)

setTopicPartitionToLeaderState

public void setTopicPartitionToLeaderState(java.lang.String topicName,
                                           int partitionId)

setTopicIngestionToFollowerState

public void setTopicIngestionToFollowerState(java.lang.String topicName,
                                             int partitionId)

getTopicPartitionIngestionStatus

public MainPartitionIngestionStatus getTopicPartitionIngestionStatus(java.lang.String topicName,
                                                                     int partitionId)

setVersionPartitionToLocalIngestion

public void setVersionPartitionToLocalIngestion(java.lang.String topicName,
                                                int partitionId)

setVersionPartitionToIsolatedIngestion

public void setVersionPartitionToIsolatedIngestion(java.lang.String topicName,
                                                   int partitionId)

cleanupTopicPartitionState

public void cleanupTopicPartitionState(java.lang.String topicName,
                                       int partitionId)

cleanupTopicState

public void cleanupTopicState(java.lang.String topicName)

getTopicPartitionCount

public long getTopicPartitionCount(java.lang.String topicName)

getTopicIngestionStatusMap

public java.util.Map<java.lang.String,MainTopicIngestionStatus> getTopicIngestionStatusMap()

getTopicPartitionLeaderStatusMap

public java.util.Map<java.lang.String,java.util.Map<java.lang.Integer,java.lang.Boolean>> getTopicPartitionLeaderStatusMap()

getForkProcessActionLock

public java.util.concurrent.locks.ReentrantReadWriteLock getForkProcessActionLock()