Package com.linkedin.davinci.ingestion

Class DefaultIngestionBackend

java.lang.Object

com.linkedin.davinci.ingestion.DefaultIngestionBackend

All Implemented Interfaces:

IngestionBackend, Closeable, AutoCloseable

public class DefaultIngestionBackend
extends Object
implements IngestionBackend

The default ingestion backend implementation.

Constructor Summary

Constructors

Constructor	Description
DefaultIngestionBackend(StorageMetadataService storageMetadataService, KafkaStoreIngestionService storeIngestionService, StorageService storageService, VeniceServerConfig serverConfig)

Method Summary

Modifier and Type	Method	Description
void	addIngestionNotifier(VeniceNotifier ingestionListener)
void	close()
CompletableFuture<Void>	dropStoragePartitionGracefully(VeniceStoreVersionConfig storeConfig, int partition, int timeoutInSeconds, boolean removeEmptyStorageEngine, String replicaId)	This method stops to subscribe the specified topic partition and delete partition data from storage.
KafkaStoreIngestionService	getStoreIngestionService()
boolean	hasCurrentVersionBootstrapping()	Check whether there are any current version bootstrapping or not.
void	killConsumptionTask(String topicName)
void	removeStorageEngine(String topicName)
void	setStorageEngineReference(String topicName, AtomicReference<StorageEngine> storageEngineReference)
void	shutdownIngestionTask(String topicName)
void	startConsumption(VeniceStoreVersionConfig storeConfig, int partition, Optional<PubSubPosition> pubSubPosition, String replicaId)
CompletableFuture<Void>	stopConsumption(VeniceStoreVersionConfig storeConfig, int partition, String replicaId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.davinci.ingestion.IngestionBackend

dropStoragePartitionGracefully

Constructor Details

DefaultIngestionBackend

public DefaultIngestionBackend(StorageMetadataService storageMetadataService,
 KafkaStoreIngestionService storeIngestionService,
 StorageService storageService,
 VeniceServerConfig serverConfig)

Method Details

startConsumption

public void startConsumption(VeniceStoreVersionConfig storeConfig,
 int partition,
 Optional<PubSubPosition> pubSubPosition,
 String replicaId)

Specified by:

startConsumption in interface IngestionBackend

stopConsumption

public CompletableFuture<Void> stopConsumption(VeniceStoreVersionConfig storeConfig,
 int partition,
 String replicaId)

Specified by:

stopConsumption in interface IngestionBackend

killConsumptionTask

public void killConsumptionTask(String topicName)

Specified by:

killConsumptionTask in interface IngestionBackend

shutdownIngestionTask

public void shutdownIngestionTask(String topicName)

Specified by:

shutdownIngestionTask in interface IngestionBackend

removeStorageEngine

public void removeStorageEngine(String topicName)

Specified by:

removeStorageEngine in interface IngestionBackend

dropStoragePartitionGracefully

public CompletableFuture<Void> dropStoragePartitionGracefully(VeniceStoreVersionConfig storeConfig,
 int partition,
 int timeoutInSeconds,
 boolean removeEmptyStorageEngine,
 String replicaId)

Description copied from interface: IngestionBackend

This method stops to subscribe the specified topic partition and delete partition data from storage.

Specified by:

dropStoragePartitionGracefully in interface IngestionBackend

Parameters:

storeConfig - Store version config

partition - Partition number to be dropped in the store version.

timeoutInSeconds - Number of seconds to wait before timeout.

removeEmptyStorageEngine - Whether to drop storage engine when dropping the last partition.

replicaId - The replica identifier for this partition.

Returns:

a future for the drop partition action.

addIngestionNotifier

public void addIngestionNotifier(VeniceNotifier ingestionListener)

Specified by:

addIngestionNotifier in interface IngestionBackend

setStorageEngineReference

public void setStorageEngineReference(String topicName,
 AtomicReference<StorageEngine> storageEngineReference)

Specified by:

setStorageEngineReference in interface IngestionBackend

hasCurrentVersionBootstrapping

public boolean hasCurrentVersionBootstrapping()

Description copied from interface: IngestionBackend

Check whether there are any current version bootstrapping or not.

Specified by:

hasCurrentVersionBootstrapping in interface IngestionBackend

getStoreIngestionService

public KafkaStoreIngestionService getStoreIngestionService()

Specified by:

getStoreIngestionService in interface IngestionBackend

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable