Package com.linkedin.davinci.consumer

Class VeniceChangelogConsumerDaVinciRecordTransformerImpl.DaVinciRecordTransformerChangelogConsumer

java.lang.Object

com.linkedin.davinci.client.DaVinciRecordTransformer<K,V,V>

com.linkedin.davinci.consumer.VeniceChangelogConsumerDaVinciRecordTransformerImpl.DaVinciRecordTransformerChangelogConsumer

All Implemented Interfaces:

Closeable, AutoCloseable

Enclosing class:

VeniceChangelogConsumerDaVinciRecordTransformerImpl<K,V>

public class VeniceChangelogConsumerDaVinciRecordTransformerImpl.DaVinciRecordTransformerChangelogConsumer
extends DaVinciRecordTransformer<K,V,V>

Constructor Summary

Constructors

Constructor	Description
DaVinciRecordTransformerChangelogConsumer(String storeName, int storeVersion, org.apache.avro.Schema keySchema, org.apache.avro.Schema inputValueSchema, org.apache.avro.Schema outputValueSchema, DaVinciRecordTransformerConfig recordTransformerConfig)

Method Summary

Modifier and Type	Method	Description
void	addMessageToBuffer(K key, V value, int partitionId, DaVinciRecordTransformerRecordMetadata recordMetadata)
void	close()
void	onHeartbeat(int partitionId, long heartbeatTimestamp)	Receive heartbeat timestamp for a partition and update latest seen time.
void	onStartVersionIngestion(boolean isCurrentVersion)	Callback invoked before consuming records for DaVinciRecordTransformer.storeVersion.
void	onVersionSwap(int currentVersion, int futureVersion, int partitionId)
void	processDelete(Lazy<K> key, int partitionId, DaVinciRecordTransformerRecordMetadata recordMetadata)	Optional callback for delete events (for example, remove from an external system).
void	processPut(Lazy<K> key, Lazy<V> value, int partitionId, DaVinciRecordTransformerRecordMetadata recordMetadata)	Callback for put/update events (for example, write to an external system).
DaVinciRecordTransformerResult<V>	transform(Lazy<K> key, Lazy<V> value, int partitionId, DaVinciRecordTransformerRecordMetadata recordMetadata)	Callback to transform records before they are stored.

Methods inherited from class com.linkedin.davinci.client.DaVinciRecordTransformer

getAlwaysBootstrapFromVersionTopic, getClassHash, getInputValueSchema, getKeySchema, getOutputValueSchema, getRecordTransformerUtility, getStoreName, getStoreRecordsInDaVinci, getStoreVersion, onEndVersionIngestion, onRecovery, prependSchemaIdToHeader, prependSchemaIdToHeader, transformAndProcessPut, useUniformInputValueSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DaVinciRecordTransformerChangelogConsumer

public DaVinciRecordTransformerChangelogConsumer(String storeName,
 int storeVersion,
 org.apache.avro.Schema keySchema,
 org.apache.avro.Schema inputValueSchema,
 org.apache.avro.Schema outputValueSchema,
 DaVinciRecordTransformerConfig recordTransformerConfig)

Method Details

onStartVersionIngestion

public void onStartVersionIngestion(boolean isCurrentVersion)

Description copied from class: DaVinciRecordTransformer

Callback invoked before consuming records for DaVinciRecordTransformer.storeVersion. Use this to open connections, create tables, or initialize resources. By default, it's a no-op.

Overrides:

onStartVersionIngestion in class DaVinciRecordTransformer<K,V,V>

Parameters:

isCurrentVersion - whether the version being started is the current serving version; if not it's a future version

transform

public DaVinciRecordTransformerResult<V> transform(Lazy<K> key,
 Lazy<V> value,
 int partitionId,
 DaVinciRecordTransformerRecordMetadata recordMetadata)

Description copied from class: DaVinciRecordTransformer

Callback to transform records before they are stored. This can be useful for tasks such as filtering out unused fields to save storage space. Result types: - SKIP: drop the record - UNCHANGED: keep the original value; useful when only callbacks/notifications are needed - TRANSFORMED: record has been transformed and should be persisted to disk

Specified by:

transform in class DaVinciRecordTransformer<K,V,V>

Parameters:

key - the key of the record to be transformed

value - the value of the record to be transformed

partitionId - what partition the record came from

recordMetadata - returns DaVinciRecordTransformerRecordMetadata if enabled in DaVinciRecordTransformerConfig, null otherwise

Returns:

DaVinciRecordTransformerResult

addMessageToBuffer

public void addMessageToBuffer(K key,
 V value,
 int partitionId,
 DaVinciRecordTransformerRecordMetadata recordMetadata)

processPut

public void processPut(Lazy<K> key,
 Lazy<V> value,
 int partitionId,
 DaVinciRecordTransformerRecordMetadata recordMetadata)

Description copied from class: DaVinciRecordTransformer

Callback for put/update events (for example, write to an external system). Invoked after DaVinciRecordTransformer.transform(Lazy, Lazy, int, DaVinciRecordTransformerRecordMetadata) when the result is UNCHANGED or TRANSFORMED. Also invoked during startup/recovery when Da Vinci replays records persisted on disk to rehydrate external systems.

Specified by:

processPut in class DaVinciRecordTransformer<K,V,V>

Parameters:

key - the key of the record to be put

value - the value of the record to be put, either the original value or the transformed value

partitionId - what partition the record came from

recordMetadata - returns DaVinciRecordTransformerRecordMetadata if enabled in DaVinciRecordTransformerConfig, null otherwise.

processDelete

public void processDelete(Lazy<K> key,
 int partitionId,
 DaVinciRecordTransformerRecordMetadata recordMetadata)

Description copied from class: DaVinciRecordTransformer

Optional callback for delete events (for example, remove from an external system). By default, this is a no-op.

Overrides:

processDelete in class DaVinciRecordTransformer<K,V,V>

Parameters:

key - the key of the record to be deleted

partitionId - what partition the record is being deleted from

recordMetadata - returns DaVinciRecordTransformerRecordMetadata if enabled in DaVinciRecordTransformerConfig, null otherwise.

onVersionSwap

public void onVersionSwap(int currentVersion,
 int futureVersion,
 int partitionId)

onHeartbeat

public void onHeartbeat(int partitionId,
 long heartbeatTimestamp)

Receive heartbeat timestamp for a partition and update latest seen time.

close

public void close()
           throws IOException

Throws:

IOException