Package com.linkedin.davinci.client

Class BlockingDaVinciRecordTransformer<K,V,O>

java.lang.Object

com.linkedin.davinci.client.DaVinciRecordTransformer<K,V,O>

com.linkedin.davinci.client.BlockingDaVinciRecordTransformer<K,V,O>

Type Parameters:

K - type of the input key

V - type of the input value

O - type of the output value

All Implemented Interfaces:

Closeable, AutoCloseable

@Experimental
public class BlockingDaVinciRecordTransformer<K,V,O>
extends DaVinciRecordTransformer<K,V,O>

This is an implementation of DaVinciRecordTransformer that implements blocking. It ensures that no puts can proceed until onStartIngestionTask finishes.

Constructor Summary

Constructors

Constructor	Description
BlockingDaVinciRecordTransformer(DaVinciRecordTransformer recordTransformer, org.apache.avro.Schema keySchema, org.apache.avro.Schema inputValueSchema, org.apache.avro.Schema outputValueSchema, boolean storeRecordsInDaVinci)

Method Summary

Modifier and Type	Method	Description
void	close()
void	onEndVersionIngestion(int currentVersion)	Lifecycle event triggered when record consumption is stopped for DaVinciRecordTransformer.storeVersion.
void	onStartVersionIngestion(boolean isCurrentVersion)	Lifecycle event triggered before consuming records for DaVinciRecordTransformer.storeVersion.
void	processDelete(Lazy<K> key)	Override this method to customize the behavior for record deletions.
void	processPut(Lazy<K> key, Lazy<O> value)	Implement this method to manage custom state outside the Da Vinci Client.
DaVinciRecordTransformerResult<O>	transform(Lazy<K> key, Lazy<V> value)	Implement this method to transform records before they are stored.

Methods inherited from class com.linkedin.davinci.client.DaVinciRecordTransformer

getClassHash, getInputValueSchema, getKeySchema, getOutputValueSchema, getRecordTransformerUtility, getStoreRecordsInDaVinci, getStoreVersion, onRecovery, prependSchemaIdToHeader, prependSchemaIdToHeader, transformAndProcessPut, useUniformInputValueSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BlockingDaVinciRecordTransformer

public BlockingDaVinciRecordTransformer(DaVinciRecordTransformer recordTransformer,
 org.apache.avro.Schema keySchema,
 org.apache.avro.Schema inputValueSchema,
 org.apache.avro.Schema outputValueSchema,
 boolean storeRecordsInDaVinci)

Method Details

transform

public DaVinciRecordTransformerResult<O> transform(Lazy<K> key,
 Lazy<V> value)

Description copied from class: DaVinciRecordTransformer

Implement this method to transform records before they are stored. This can be useful for tasks such as filtering out unused fields to save storage space.

Specified by:

transform in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be transformed

value - the value of the record to be transformed

Returns:

DaVinciRecordTransformerResult

processPut

public void processPut(Lazy<K> key,
 Lazy<O> value)

Description copied from class: DaVinciRecordTransformer

Implement this method to manage custom state outside the Da Vinci Client.

Specified by:

processPut in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be put

value - the value of the record to be put, derived from the output of DaVinciRecordTransformer.transform(Lazy key, Lazy value)

processDelete

public void processDelete(Lazy<K> key)

Description copied from class: DaVinciRecordTransformer

Override this method to customize the behavior for record deletions. For example, you can use this method to delete records from a custom storage outside the Da Vinci Client. By default, it performs no operation.

Overrides:

processDelete in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be deleted

onStartVersionIngestion

public void onStartVersionIngestion(boolean isCurrentVersion)

Description copied from class: DaVinciRecordTransformer

Lifecycle event triggered before consuming records for DaVinciRecordTransformer.storeVersion. Use this method to perform setup operations such as opening database connections or creating tables. By default, it performs no operation.

Overrides:

onStartVersionIngestion in class DaVinciRecordTransformer<K,V,O>

onEndVersionIngestion

public void onEndVersionIngestion(int currentVersion)

Description copied from class: DaVinciRecordTransformer

Lifecycle event triggered when record consumption is stopped for DaVinciRecordTransformer.storeVersion. Use this method to perform cleanup operations such as closing database connections or dropping tables. By default, it performs no operation.

Overrides:

onEndVersionIngestion in class DaVinciRecordTransformer<K,V,O>

close

public void close()
           throws IOException

Throws:

IOException