Package com.linkedin.davinci.store.view

Class VeniceViewWriter

java.lang.Object

com.linkedin.venice.views.VeniceView

com.linkedin.davinci.store.view.VeniceViewWriter

Direct Known Subclasses:

ChangeCaptureViewWriter

public abstract class VeniceViewWriter
extends VeniceView

This class is kept separate from the VeniceView class to not leak certain concepts that currently exclusively reside in the server to other components. This class is created per LeaderFollowerStoreIngestionTask per view and is invoked to processRecords in the ActiveActiveStoreIngestionTask as it's consumed from Kafka and after MergeConflict resolution has run. It's then invoked prior to committing to local persistent storage or local VT production in order to keep integrity of the data in various failure scenarios. This implies that repeat calls on the same data for processRecords can happen and there should be some protection from that being an issue in different view implementations.

Field Summary

Fields inherited from class com.linkedin.venice.views.VeniceView

props, store, VIEW_TOPIC_SEPARATOR, viewParameters

Constructor Summary

Constructors

Constructor	Description
VeniceViewWriter(VeniceConfigLoader props, Store store, org.apache.avro.Schema keySchema, java.util.Map<java.lang.String,java.lang.String> extraViewParameters)

Method Summary

Modifier and Type	Method	Description
void	processControlMessage(ControlMessage controlMessage, int partition, PartitionConsumptionState partitionConsumptionState, int version)	Called when the server encounters a control message.
java.util.concurrent.CompletableFuture<PubSubProduceResult>	processRecord(java.nio.ByteBuffer newValue, java.nio.ByteBuffer oldValue, byte[] key, int version, int newValueSchemaId, int oldValueSchemaId, org.apache.avro.generic.GenericRecord replicationMetadataRecord)	To be called as a given ingestion task consumes each record.

Methods inherited from class com.linkedin.venice.views.VeniceView

close, getTopicNamesAndConfigsForVersion, getWriterClassName, isViewTopic, parseStoreFromViewTopic, parseVersionFromViewTopic, validateConfigs

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VeniceViewWriter

public VeniceViewWriter(VeniceConfigLoader props,
                        Store store,
                        org.apache.avro.Schema keySchema,
                        java.util.Map<java.lang.String,java.lang.String> extraViewParameters)

Method Detail

processRecord

public java.util.concurrent.CompletableFuture<PubSubProduceResult> processRecord(java.nio.ByteBuffer newValue,
                                                                                 java.nio.ByteBuffer oldValue,
                                                                                 byte[] key,
                                                                                 int version,
                                                                                 int newValueSchemaId,
                                                                                 int oldValueSchemaId,
                                                                                 org.apache.avro.generic.GenericRecord replicationMetadataRecord)

To be called as a given ingestion task consumes each record. This is called prior to writing to a VT or to persistent storage.

Parameters:

newValue - the incoming fully specified value which hasn't yet been committed to Venice

oldValue - the previous value which has already been locally committed to Venice for the given key

key - the key of the record that designates newValue and oldValue

version - the version of the store taking this record

newValueSchemaId - the schemaId of the incoming record

oldValueSchemaId - the schemaId of the old record

replicationMetadataRecord - the associated RMD for the incoming record.

processControlMessage

public void processControlMessage(ControlMessage controlMessage,
                                  int partition,
                                  PartitionConsumptionState partitionConsumptionState,
                                  int version)

Called when the server encounters a control message. There isn't (today) a strict ordering on if the rest of the server alters it's state completely or not based on the incoming control message relative to the given view. TODO: Today this is only invoked for VERSION_SWAP control message, but we may in the future call this method for all control messages so that certain view types can act accordingly.

Parameters:

controlMessage - the control message we're processing

partition - the partition this control message was delivered to

partitionConsumptionState - the pcs of the consuming node

version - the store version that received this message