Package com.linkedin.davinci.store.view

Class MaterializedViewWriter

java.lang.Object

com.linkedin.venice.views.VeniceView

com.linkedin.davinci.store.view.VeniceViewWriter

com.linkedin.davinci.store.view.MaterializedViewWriter

public class MaterializedViewWriter
extends VeniceViewWriter

Materialized view writer is responsible for processing input records from the version topic and write them to the materialized view topic based on parameters defined in MaterializedViewParameters. This writer has its own VeniceWriter.

Field Summary

Fields inherited from class com.linkedin.davinci.store.view.VeniceViewWriter

isNearlineProducerCompressionEnabled, nearlineProducerCountPerWriter, version, versionNumber

Fields inherited from class com.linkedin.venice.views.VeniceView

props, storeName, VIEW_TOPIC_SEPARATOR, viewParameters

Constructor Summary

Constructors

Constructor	Description
MaterializedViewWriter(VeniceConfigLoader props, Version version, org.apache.avro.Schema keySchema, Map<String,String> extraViewParameters)

Method Summary

Modifier and Type	Method	Description
String	getWriterClassName()	Implementations should return the fully specified class name for the component VeniceViewWriter implementation.
void	processControlMessage(KafkaKey kafkaKey, KafkaMessageEnvelope kafkaMessageEnvelope, ControlMessage controlMessage, int partition, PartitionConsumptionState partitionConsumptionState)	Called when the server encounters a control message.
CompletableFuture<PubSubProduceResult>	processRecord(ByteBuffer newValue, byte[] key, int newValueSchemaId)	To be called as a given ingestion task consumes each record.
CompletableFuture<PubSubProduceResult>	processRecord(ByteBuffer newValue, ByteBuffer oldValue, byte[] key, int newValueSchemaId, int oldValueSchemaId, org.apache.avro.generic.GenericRecord replicationMetadataRecord)	To be called as a given ingestion task consumes each record.

Methods inherited from class com.linkedin.davinci.store.view.VeniceViewWriter

setProducerOptimizations

Methods inherited from class com.linkedin.venice.views.VeniceView

close, getTopicNamesAndConfigsForVersion, getWriterOptionsBuilder, isViewTopic, parseStoreFromViewTopic, parseVersionFromViewTopic, validateConfigs

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

MaterializedViewWriter

public MaterializedViewWriter(VeniceConfigLoader props,
 Version version,
 org.apache.avro.Schema keySchema,
 Map<String,String> extraViewParameters)

Method Details

processRecord

public CompletableFuture<PubSubProduceResult> processRecord(ByteBuffer newValue,
 ByteBuffer oldValue,
 byte[] key,
 int newValueSchemaId,
 int oldValueSchemaId,
 org.apache.avro.generic.GenericRecord replicationMetadataRecord)

Description copied from class: VeniceViewWriter

To be called as a given ingestion task consumes each record. This is called prior to writing to a VT or to persistent storage.

Specified by:

processRecord in class VeniceViewWriter

Parameters:

newValue - the incoming fully specified value which hasn't yet been committed to Venice

oldValue - the previous value which has already been locally committed to Venice for the given key

key - the key of the record that designates newValue and oldValue

newValueSchemaId - the schemaId of the incoming record

oldValueSchemaId - the schemaId of the old record

replicationMetadataRecord - the associated RMD for the incoming record.

processRecord

public CompletableFuture<PubSubProduceResult> processRecord(ByteBuffer newValue,
 byte[] key,
 int newValueSchemaId)

Description copied from class: VeniceViewWriter

To be called as a given ingestion task consumes each record. This is called prior to writing to a VT or to persistent storage.

Specified by:

processRecord in class VeniceViewWriter

Parameters:

newValue - the incoming fully specified value which hasn't yet been committed to Venice

key - the key of the record that designates newValue and oldValue

newValueSchemaId - the schemaId of the incoming record

processControlMessage

public void processControlMessage(KafkaKey kafkaKey,
 KafkaMessageEnvelope kafkaMessageEnvelope,
 ControlMessage controlMessage,
 int partition,
 PartitionConsumptionState partitionConsumptionState)

Description copied from class: VeniceViewWriter

Called when the server encounters a control message. There isn't (today) a strict ordering on if the rest of the server alters it's state completely or not based on the incoming control message relative to the given view. Different view types may be interested in different control messages and act differently. The corresponding view writer should implement this method accordingly.

Overrides:

processControlMessage in class VeniceViewWriter

Parameters:

kafkaKey - the corresponding kafka key of this control message

kafkaMessageEnvelope - the corresponding kafka message envelope of this control message

controlMessage - the control message we're processing

partition - the partition this control message was delivered to

partitionConsumptionState - the pcs of the consuming node

getWriterClassName

public String getWriterClassName()

Description copied from class: VeniceView

Implementations should return the fully specified class name for the component VeniceViewWriter implementation.

Overrides:

getWriterClassName in class VeniceView

Returns:

returns the className of the writer which should be instantiated.