Package com.linkedin.venice.serialization.avro

Enum AvroProtocolDefinition

  • All Implemented Interfaces:
    java.io.Serializable, java.lang.Comparable<AvroProtocolDefinition>
    public enum AvroProtocolDefinition
extends java.lang.Enum<AvroProtocolDefinition>
    This enum lays out the basic specs of the various stateful protocols used in Venice. Having these definitions in a single place makes it easy to ensure that magic bytes are defined only once and do not conflict with each other.

    • Enum Constant Detail

      • KAFKA_MESSAGE_ENVELOPE

        public static final AvroProtocolDefinition KAFKA_MESSAGE_ENVELOPE
        Used for the Kafka topics, including the main data topics as well as the admin topic.

      • PARTITION_STATE

        public static final AvroProtocolDefinition PARTITION_STATE
        Used to persist the state of a partition in Storage Nodes, including offset, Data Ingest Validation state, etc.

      • STORE_VERSION_STATE

        public static final AvroProtocolDefinition STORE_VERSION_STATE
        Used to persist state related to a store-version, including Start of Buffer Replay offsets and whether the input is sorted.

      • PUSH_JOB_DETAILS

        public static final AvroProtocolDefinition PUSH_JOB_DETAILS
        Used to encode push job details records to be written to the PushJobDetails system store.

      • ADMIN_OPERATION

        public static final AvroProtocolDefinition ADMIN_OPERATION
        Used to encode metadata changes about the system as a whole. Records of this type are propagated via the "admin channel" special Kafka topic. It is stored in the Put of a KafkaMessageEnvelope, and thus leverages the envelope for versioning. TODO: Move AdminOperation to venice-common module so that we can properly reference it here.

      • CHUNK

        public static final AvroProtocolDefinition CHUNK
        Single chunk of a large multi-chunk value. Just a bunch of bytes. Uses a negative protocol version in order to avoid clashing with user-defined schemas.

      • CHUNKED_VALUE_MANIFEST

        public static final AvroProtocolDefinition CHUNKED_VALUE_MANIFEST
        Used to encode the manifest of a multi-chunk large value. It is stored in the Put of a KafkaMessageEnvelope, and thus leverages the envelope for versioning. Uses a negative protocol version in order to avoid clashing with user-defined schemas.

      • CHUNKED_KEY_SUFFIX

        public static final AvroProtocolDefinition CHUNKED_KEY_SUFFIX
        Suffix appended to the end of all keys in a store-version where chunking is enabled. This protocol is actually un-evolvable.

      • INGESTION_TASK_COMMAND

        public static final AvroProtocolDefinition INGESTION_TASK_COMMAND
        Used to encode various kinds of ingestion task commands, which are used to control ingestion task in child process.

      • INGESTION_TASK_REPORT

        public static final AvroProtocolDefinition INGESTION_TASK_REPORT
        Used to encode status of ingestion task, that are reported backed from child process to Storage Node / Da Vinci backend.

      • INGESTION_METRICS_REPORT

        public static final AvroProtocolDefinition INGESTION_METRICS_REPORT
        Used to encode metrics collected from ingestion task, that are reported backed from child process to Storage Node / Da Vinci backend.

      • INGESTION_STORAGE_METADATA

        public static final AvroProtocolDefinition INGESTION_STORAGE_METADATA
        Used to encode storage metadata updates that are reported backed from Storage Node / Da Vinci backend to child process.

      • PROCESS_SHUTDOWN_COMMAND

        public static final AvroProtocolDefinition PROCESS_SHUTDOWN_COMMAND
        Used to encode various kinds of ingestion task commands, which are used to control ingestion task in child process.

      • PUBSUB_POSITION_WIRE_FORMAT

        public static final AvroProtocolDefinition PUBSUB_POSITION_WIRE_FORMAT
        Used to encode the position of a PubSub message.

      • LOADED_STORE_USER_PARTITION_MAPPING

        public static final AvroProtocolDefinition LOADED_STORE_USER_PARTITION_MAPPING
        Used to retrieve the loaded store partition mapping in the isolated process. In theory, we don't need to use magicByte for the communication with II process.

      • METADATA_SYSTEM_SCHEMA_STORE_KEY

        public static final AvroProtocolDefinition METADATA_SYSTEM_SCHEMA_STORE_KEY
        Key schema for metadata system store.

      • METADATA_SYSTEM_SCHEMA_STORE

        public static final AvroProtocolDefinition METADATA_SYSTEM_SCHEMA_STORE
        Value schema for metadata system store.

      • PUSH_STATUS_SYSTEM_SCHEMA_STORE_KEY

        public static final AvroProtocolDefinition PUSH_STATUS_SYSTEM_SCHEMA_STORE_KEY
        Key schema for push status system store.

      • PUSH_STATUS_SYSTEM_SCHEMA_STORE

        public static final AvroProtocolDefinition PUSH_STATUS_SYSTEM_SCHEMA_STORE
        Value schema for push status system store.

      • PARTICIPANT_MESSAGE_SYSTEM_STORE_VALUE

        public static final AvroProtocolDefinition PARTICIPANT_MESSAGE_SYSTEM_STORE_VALUE
        Value schema for participant system stores.

      • SERVER_ADMIN_RESPONSE

        public static final AvroProtocolDefinition SERVER_ADMIN_RESPONSE
        Response record for admin request.

      • SERVER_METADATA_RESPONSE

        public static final AvroProtocolDefinition SERVER_METADATA_RESPONSE
        Response record for metadata fetch request.

      • RECORD_CHANGE_EVENT

        public static final AvroProtocolDefinition RECORD_CHANGE_EVENT
        Value schema for change capture event. TODO: Figure out a way to pull in protocol from different view class.

    • Field Detail

      • currentProtocolVersion

        public final java.util.Optional<java.lang.Integer> currentProtocolVersion
        The protocol version that is currently in use. Typically, this should be the greatest number available, but that may not necessarily be true if some new experimental version of a protocol is being designed and is not yet rolled out.

    • Method Detail

      • values

        public static AvroProtocolDefinition[] values()
        Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows: 
        for (AvroProtocolDefinition c : AvroProtocolDefinition.values())
    System.out.println(c);
        Returns:
        an array containing the constants of this enum type, in the order they are declared

      • valueOf

        public static AvroProtocolDefinition valueOf​(java.lang.String name)
        Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)
        Parameters:
        name - the name of the enum constant to be returned.
        Returns:
        the enum constant with the specified name
        Throws:
        java.lang.IllegalArgumentException - if this enum type has no constant with the specified name
        java.lang.NullPointerException - if the argument is null

      • getCurrentProtocolVersion

        public int getCurrentProtocolVersion()

      • getCurrentProtocolVersionSchema

        public org.apache.avro.Schema getCurrentProtocolVersionSchema()

      • getMagicByte

        public java.util.Optional<java.lang.Byte> getMagicByte()

      • getClassName

        public java.lang.String getClassName()

      • getSystemStoreName

        public java.lang.String getSystemStoreName()