Package com.linkedin.davinci.client

Class BlockingDaVinciRecordTransformer<K,V,O>

java.lang.Object

com.linkedin.davinci.client.DaVinciRecordTransformer<K,V,O>

com.linkedin.davinci.client.BlockingDaVinciRecordTransformer<K,V,O>

Type Parameters:

K - type of the input key

V - type of the input value

O - type of the output value

All Implemented Interfaces:

Closeable, AutoCloseable

@Experimental
public class BlockingDaVinciRecordTransformer<K,V,O>
extends DaVinciRecordTransformer<K,V,O>

This is an implementation of DaVinciRecordTransformer that implements blocking. It ensures that no puts can proceed until onStartIngestionTask finishes.

Constructor Summary

Constructors

Constructor	Description
BlockingDaVinciRecordTransformer(DaVinciRecordTransformer recordTransformer, org.apache.avro.Schema keySchema, org.apache.avro.Schema inputValueSchema, org.apache.avro.Schema outputValueSchema, DaVinciRecordTransformerConfig recordTransformerConfig)

Method Summary

Modifier and Type	Method	Description
void	close()
void	internalOnRecovery(AbstractStorageEngine storageEngine, int partitionId, InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer, Lazy<VeniceCompressor> compressor)
void	onEndVersionIngestion(int currentVersion)	Lifecycle event triggered when record consumption is stopped for DaVinciRecordTransformer.storeVersion.
void	onStartVersionIngestion(boolean isCurrentVersion)	Lifecycle event triggered before consuming records for DaVinciRecordTransformer.storeVersion.
void	onVersionSwap(int currentVersion, int futureVersion, int partitionId)	Lifecycle event triggered when a version swap is detected for partitionId It is used for DVRT CDC.
void	processDelete(Lazy<K> key, int partitionId)	Override this method to customize the behavior for record deletions.
void	processPut(Lazy<K> key, Lazy<O> value, int partitionId)	Implement this method to manage custom state outside the Da Vinci Client.
DaVinciRecordTransformerResult<O>	transform(Lazy<K> key, Lazy<V> value, int partitionId)	Implement this method to transform records before they are stored.

Methods inherited from class com.linkedin.davinci.client.DaVinciRecordTransformer

getAlwaysBootstrapFromVersionTopic, getClassHash, getInputValueSchema, getKeySchema, getOutputValueSchema, getRecordTransformerUtility, getStoreRecordsInDaVinci, getStoreVersion, onRecovery, prependSchemaIdToHeader, prependSchemaIdToHeader, transformAndProcessPut, useUniformInputValueSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BlockingDaVinciRecordTransformer

public BlockingDaVinciRecordTransformer(DaVinciRecordTransformer recordTransformer,
 org.apache.avro.Schema keySchema,
 org.apache.avro.Schema inputValueSchema,
 org.apache.avro.Schema outputValueSchema,
 DaVinciRecordTransformerConfig recordTransformerConfig)

Method Details

transform

public DaVinciRecordTransformerResult<O> transform(Lazy<K> key,
 Lazy<V> value,
 int partitionId)

Description copied from class: DaVinciRecordTransformer

Implement this method to transform records before they are stored. This can be useful for tasks such as filtering out unused fields to save storage space.

Specified by:

transform in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be transformed

value - the value of the record to be transformed

partitionId - what partition the record came from

Returns:

DaVinciRecordTransformerResult

processPut

public void processPut(Lazy<K> key,
 Lazy<O> value,
 int partitionId)

Description copied from class: DaVinciRecordTransformer

Implement this method to manage custom state outside the Da Vinci Client.

Specified by:

processPut in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be put

value - the value of the record to be put, derived from the output of DaVinciRecordTransformer.transform(Lazy key, Lazy value, int partitionId)

partitionId - what partition the record came from

processDelete

public void processDelete(Lazy<K> key,
 int partitionId)

Description copied from class: DaVinciRecordTransformer

Override this method to customize the behavior for record deletions. For example, you can use this method to delete records from a custom storage outside the Da Vinci Client. By default, it performs no operation.

Overrides:

processDelete in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be deleted

partitionId - what partition the record is being deleted from

onStartVersionIngestion

public void onStartVersionIngestion(boolean isCurrentVersion)

Description copied from class: DaVinciRecordTransformer

Lifecycle event triggered before consuming records for DaVinciRecordTransformer.storeVersion. Use this method to perform setup operations such as opening database connections or creating tables. By default, it performs no operation.

Overrides:

onStartVersionIngestion in class DaVinciRecordTransformer<K,V,O>

onEndVersionIngestion

public void onEndVersionIngestion(int currentVersion)

Description copied from class: DaVinciRecordTransformer

Lifecycle event triggered when record consumption is stopped for DaVinciRecordTransformer.storeVersion. Use this method to perform cleanup operations such as closing database connections or dropping tables. By default, it performs no operation.

Overrides:

onEndVersionIngestion in class DaVinciRecordTransformer<K,V,O>

onVersionSwap

public void onVersionSwap(int currentVersion,
 int futureVersion,
 int partitionId)

Lifecycle event triggered when a version swap is detected for partitionId It is used for DVRT CDC.

internalOnRecovery

public void internalOnRecovery(AbstractStorageEngine storageEngine,
 int partitionId,
 InternalAvroSpecificSerializer<PartitionState> partitionStateSerializer,
 Lazy<VeniceCompressor> compressor)

close

public void close()
           throws IOException

Throws:

IOException