Package com.linkedin.davinci.store.view

Class MaterializedViewWriter

java.lang.Object
com.linkedin.venice.views.VeniceView
com.linkedin.davinci.store.view.VeniceViewWriter
com.linkedin.davinci.store.view.MaterializedViewWriter
public class MaterializedViewWriter extends VeniceViewWriter
Materialized view writer is responsible for processing input records from the version topic and write them to the materialized view topic based on parameters defined in MaterializedViewParameters. This writer has its own VeniceWriter.

  • Constructor Details

  • Method Details

    • processRecord

      public CompletableFuture<Void> processRecord(ByteBuffer newValue, ByteBuffer oldValue, byte[] key, int newValueSchemaId, int oldValueSchemaId, org.apache.avro.generic.GenericRecord replicationMetadataRecord, Lazy<org.apache.avro.generic.GenericRecord> valueProvider)
      Description copied from class: VeniceViewWriter
      To be called as a given ingestion task consumes each record. This is called prior to writing to a VT or to persistent storage.
      Specified by:
      processRecord in class VeniceViewWriter
      Parameters:
      newValue - the incoming fully specified value which hasn't yet been committed to Venice
      oldValue - the previous value which has already been locally committed to Venice for the given key
      key - the key of the record that designates newValue and oldValue
      newValueSchemaId - the schemaId of the incoming record
      oldValueSchemaId - the schemaId of the old record
      replicationMetadataRecord - the associated RMD for the incoming record.
      valueProvider - to provide the corresponding deserialized newValue for PUT and UPDATE or the old value for the given key for DELETE.

    • processRecord

      public CompletableFuture<Void> processRecord(ByteBuffer newValue, byte[] key, int newValueSchemaId, boolean isChunkedKey, Lazy<org.apache.avro.generic.GenericRecord> newValueProvider)
      Before we have proper chunking support for view writers we assume that even when chunking is enabled the actual k/v will not be chunked. This way we don't need to worry about how to ensure all the chunks are forwarded to the same view partition during NR's pass-through mode. Proper chunking support can leverage message footer populated by the CompositeVeniceWriter in VPJ (write to views first and then VT) to figure out which view partition to forward the chunks to. Another alternative is trigger the view writer write upon receiving the manifest, and we will assemble and re-chunk.
      Specified by:
      processRecord in class VeniceViewWriter
      Parameters:
      newValue - the incoming fully specified value which hasn't yet been committed to Venice
      key - the key of the record that designates newValue and oldValue
      newValueSchemaId - the schemaId of the incoming record
      isChunkedKey - is the key already serialized with KeyWithChunkingSuffixSerializer
      newValueProvider - to provide the deserialized new value

    • processControlMessage

      public void processControlMessage(KafkaKey kafkaKey, KafkaMessageEnvelope kafkaMessageEnvelope, ControlMessage controlMessage, int partition, PartitionConsumptionState partitionConsumptionState)
      Description copied from class: VeniceViewWriter
      Called when the server encounters a control message. There isn't (today) a strict ordering on if the rest of the server alters it's state completely or not based on the incoming control message relative to the given view. Different view types may be interested in different control messages and act differently. The corresponding view writer should implement this method accordingly.
      Overrides:
      processControlMessage in class VeniceViewWriter
      Parameters:
      kafkaKey - the corresponding kafka key of this control message
      kafkaMessageEnvelope - the corresponding kafka message envelope of this control message
      controlMessage - the control message we're processing
      partition - the partition this control message was delivered to
      partitionConsumptionState - the pcs of the consuming node

    • getWriterClassName

      public String getWriterClassName()
      Description copied from class: VeniceView
      Implementations should return the fully specified class name for the component VeniceViewWriter implementation.
      Overrides:
      getWriterClassName in class VeniceView
      Returns:
      returns the className of the writer which should be instantiated.

    • isComplexVenicePartitioner

      public boolean isComplexVenicePartitioner()