Package com.linkedin.davinci.ingestion

Class DefaultIngestionBackend

java.lang.Object

com.linkedin.davinci.ingestion.DefaultIngestionBackend

All Implemented Interfaces:

IngestionBackend, java.io.Closeable, java.lang.AutoCloseable

Direct Known Subclasses:

IsolatedIngestionBackend

public class DefaultIngestionBackend
extends java.lang.Object
implements IngestionBackend

The default ingestion backend implementation. Ingestion will be done in the same JVM as the application.

Constructor Summary

Constructors

Constructor	Description
DefaultIngestionBackend(StorageMetadataService storageMetadataService, KafkaStoreIngestionService storeIngestionService, StorageService storageService, BlobTransferManager blobTransferManager, VeniceServerConfig serverConfig)

Method Summary

Modifier and Type	Method	Description
void	addIngestionNotifier(VeniceNotifier ingestionListener)
void	close()
void	dropStoragePartitionGracefully(VeniceStoreVersionConfig storeConfig, int partition, int timeoutInSeconds, boolean removeEmptyStorageEngine)	This method stops to subscribe the specified topic partition and delete partition data from storage.
KafkaStoreIngestionService	getStoreIngestionService()
void	killConsumptionTask(java.lang.String topicName)
void	removeStorageEngine(java.lang.String topicName)
void	setStorageEngineReference(java.lang.String topicName, java.util.concurrent.atomic.AtomicReference<AbstractStorageEngine> storageEngineReference)
void	shutdownIngestionTask(java.lang.String topicName)
void	startConsumption(VeniceStoreVersionConfig storeConfig, int partition)
java.util.concurrent.CompletableFuture<java.lang.Void>	stopConsumption(VeniceStoreVersionConfig storeConfig, int partition)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.davinci.ingestion.IngestionBackend

dropStoragePartitionGracefully

Constructor Detail

DefaultIngestionBackend

public DefaultIngestionBackend(StorageMetadataService storageMetadataService,
                               KafkaStoreIngestionService storeIngestionService,
                               StorageService storageService,
                               BlobTransferManager blobTransferManager,
                               VeniceServerConfig serverConfig)

Method Detail

startConsumption

public void startConsumption(VeniceStoreVersionConfig storeConfig,
                             int partition)

Specified by:

startConsumption in interface IngestionBackend

stopConsumption

public java.util.concurrent.CompletableFuture<java.lang.Void> stopConsumption(VeniceStoreVersionConfig storeConfig,
                                                                              int partition)

Specified by:

stopConsumption in interface IngestionBackend

killConsumptionTask

public void killConsumptionTask(java.lang.String topicName)

Specified by:

killConsumptionTask in interface IngestionBackend

shutdownIngestionTask

public void shutdownIngestionTask(java.lang.String topicName)

Specified by:

shutdownIngestionTask in interface IngestionBackend

removeStorageEngine

public void removeStorageEngine(java.lang.String topicName)

Specified by:

removeStorageEngine in interface IngestionBackend

dropStoragePartitionGracefully

public void dropStoragePartitionGracefully(VeniceStoreVersionConfig storeConfig,
                                           int partition,
                                           int timeoutInSeconds,
                                           boolean removeEmptyStorageEngine)

Description copied from interface: IngestionBackend

This method stops to subscribe the specified topic partition and delete partition data from storage.

Specified by:

dropStoragePartitionGracefully in interface IngestionBackend

Parameters:

storeConfig - Store version config

partition - Partition number to be dropped in the store version.

timeoutInSeconds - Number of seconds to wait before timeout.

removeEmptyStorageEngine - Whether to drop storage engine when dropping the last partition.

addIngestionNotifier

public void addIngestionNotifier(VeniceNotifier ingestionListener)

Specified by:

addIngestionNotifier in interface IngestionBackend

setStorageEngineReference

public void setStorageEngineReference(java.lang.String topicName,
                                      java.util.concurrent.atomic.AtomicReference<AbstractStorageEngine> storageEngineReference)

Specified by:

setStorageEngineReference in interface IngestionBackend

getStoreIngestionService

public KafkaStoreIngestionService getStoreIngestionService()

Specified by:

getStoreIngestionService in interface IngestionBackend

close

public void close()

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable