Package com.linkedin.davinci.ingestion.main

Class MainIngestionMonitorService

java.lang.Object

com.linkedin.venice.service.AbstractVeniceService

com.linkedin.davinci.ingestion.main.MainIngestionMonitorService

All Implemented Interfaces:

Closeable, AutoCloseable

public class MainIngestionMonitorService
extends AbstractVeniceService

This class is the listener service in main process which handles various kinds of reports sent from isolated ingestion service. MainIngestionMonitorService itself is a Netty based server implementation, and the main report handling logics happens in MainIngestionReportHandler. Besides reports handling, it also maintains two executor services to send heartbeat check and collect metrics to/from child process. Also, it maintains status for all the ongoing/completed topic partition ingestion tasks, which helps IsolatedIngestionBackend to check which process a topic partition storage is located, as well as status recovery when child process crashed and restarted.

Nested Class Summary

Nested classes/interfaces inherited from class com.linkedin.venice.service.AbstractVeniceService

AbstractVeniceService.ServiceState

Field Summary

Fields inherited from class com.linkedin.venice.service.AbstractVeniceService

logger, serviceState

Constructor Summary

Constructors

Constructor	Description
MainIngestionMonitorService(IsolatedIngestionBackend ingestionBackend, VeniceConfigLoader configLoader)

Method Summary

Modifier and Type	Method	Description
void	addIngestionNotifier(VeniceNotifier ingestionListener)
void	addPushStatusNotifier(VeniceNotifier pushStatusNotifier)
void	cleanupTopicPartitionState(String topicName, int partitionId)
void	cleanupTopicState(String topicName)
List<VeniceNotifier>	getIngestionNotifier()
PubSubPositionDeserializer	getPubSubPositionDeserializer()
List<VeniceNotifier>	getPushStatusNotifierList()
MainIngestionStorageMetadataService	getStorageMetadataService()
KafkaStoreIngestionService	getStoreIngestionService()
Map<String,MainTopicIngestionStatus>	getTopicIngestionStatusMap()
long	getTopicPartitionCount(String topicName)
MainPartitionIngestionStatus	getTopicPartitionIngestionStatus(String topicName, int partitionId)
void	setMetricsRepository(io.tehuti.metrics.MetricsRepository metricsRepository)
void	setStorageMetadataService(MainIngestionStorageMetadataService storageMetadataService)
void	setStoreIngestionService(KafkaStoreIngestionService storeIngestionService)
void	setVersionPartitionToIsolatedIngestion(String topicName, int partitionId)
void	setVersionPartitionToLocalIngestion(String topicName, int partitionId)
boolean	startInner()
void	stopInner()

Methods inherited from class com.linkedin.venice.service.AbstractVeniceService

close, getName, isRunning, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

MainIngestionMonitorService

public MainIngestionMonitorService(IsolatedIngestionBackend ingestionBackend,
 VeniceConfigLoader configLoader)

Method Details

startInner

public boolean startInner()
                   throws Exception

Specified by:

startInner in class AbstractVeniceService

Returns:

true if the service is completely started, false if it is still starting asynchronously (in this case, it is the implementer's responsibility to set AbstractVeniceService.serviceState to AbstractVeniceService.ServiceState.STARTED upon completion of the async work).

Throws:

Exception

stopInner

public void stopInner()
               throws Exception

Specified by:

stopInner in class AbstractVeniceService

Throws:

Exception

addIngestionNotifier

public void addIngestionNotifier(VeniceNotifier ingestionListener)

getIngestionNotifier

public List<VeniceNotifier> getIngestionNotifier()

getPubSubPositionDeserializer

public PubSubPositionDeserializer getPubSubPositionDeserializer()

addPushStatusNotifier

public void addPushStatusNotifier(VeniceNotifier pushStatusNotifier)

getPushStatusNotifierList

public List<VeniceNotifier> getPushStatusNotifierList()

setMetricsRepository

public void setMetricsRepository(io.tehuti.metrics.MetricsRepository metricsRepository)

setStorageMetadataService

public void setStorageMetadataService(MainIngestionStorageMetadataService storageMetadataService)

getStorageMetadataService

public MainIngestionStorageMetadataService getStorageMetadataService()

setStoreIngestionService

public void setStoreIngestionService(KafkaStoreIngestionService storeIngestionService)

getStoreIngestionService

public KafkaStoreIngestionService getStoreIngestionService()

getTopicPartitionIngestionStatus

public MainPartitionIngestionStatus getTopicPartitionIngestionStatus(String topicName,
 int partitionId)

setVersionPartitionToLocalIngestion

public void setVersionPartitionToLocalIngestion(String topicName,
 int partitionId)

setVersionPartitionToIsolatedIngestion

public void setVersionPartitionToIsolatedIngestion(String topicName,
 int partitionId)

cleanupTopicPartitionState

public void cleanupTopicPartitionState(String topicName,
 int partitionId)

cleanupTopicState

public void cleanupTopicState(String topicName)

getTopicPartitionCount

public long getTopicPartitionCount(String topicName)

getTopicIngestionStatusMap

public Map<String,MainTopicIngestionStatus> getTopicIngestionStatusMap()