Class VeniceViewWriter

  • Direct Known Subclasses:

    public abstract class VeniceViewWriter
    extends VeniceView
    This class is kept separate from the VeniceView class to not leak certain concepts that currently exclusively reside in the server to other components. This class is created per LeaderFollowerStoreIngestionTask per view and is invoked to processRecords in the ActiveActiveStoreIngestionTask as it's consumed from Kafka and after MergeConflict resolution has run. It's then invoked prior to committing to local persistent storage or local VT production in order to keep integrity of the data in various failure scenarios. This implies that repeat calls on the same data for processRecords can happen and there should be some protection from that being an issue in different view implementations.
    • Constructor Detail

      • VeniceViewWriter

        public VeniceViewWriter​(VeniceConfigLoader props,
                                Store store,
                                org.apache.avro.Schema keySchema,
                                java.util.Map<java.lang.String,​java.lang.String> extraViewParameters)
    • Method Detail

      • processRecord

        public java.util.concurrent.CompletableFuture<PubSubProduceResult> processRecord​(java.nio.ByteBuffer newValue,
                                                                                         java.nio.ByteBuffer oldValue,
                                                                                         byte[] key,
                                                                                         int version,
                                                                                         int newValueSchemaId,
                                                                                         int oldValueSchemaId,
                                                                                         org.apache.avro.generic.GenericRecord replicationMetadataRecord)
        To be called as a given ingestion task consumes each record. This is called prior to writing to a VT or to persistent storage.
        newValue - the incoming fully specified value which hasn't yet been committed to Venice
        oldValue - the previous value which has already been locally committed to Venice for the given key
        key - the key of the record that designates newValue and oldValue
        version - the version of the store taking this record
        newValueSchemaId - the schemaId of the incoming record
        oldValueSchemaId - the schemaId of the old record
        replicationMetadataRecord - the associated RMD for the incoming record.
      • processControlMessage

        public void processControlMessage​(ControlMessage controlMessage,
                                          int partition,
                                          PartitionConsumptionState partitionConsumptionState,
                                          int version)
        Called when the server encounters a control message. There isn't (today) a strict ordering on if the rest of the server alters it's state completely or not based on the incoming control message relative to the given view. TODO: Today this is only invoked for VERSION_SWAP control message, but we may in the future call this method for all control messages so that certain view types can act accordingly.
        controlMessage - the control message we're processing
        partition - the partition this control message was delivered to
        partitionConsumptionState - the pcs of the consuming node
        version - the store version that received this message