Package com.linkedin.venice.hadoop.task.datawriter

Class ExternalStorageRecord

java.lang.Object
com.linkedin.venice.hadoop.task.datawriter.ExternalStorageRecord
public final class ExternalStorageRecord extends Object
Immutable carrier passed to ExternalStorageWriter.batchPut(java.util.List). Holds one record's serialized key and a value blob whose layout matches Venice's on-disk format in RocksDB: 
   value = [4-byte big-endian value schema id] [compressed Avro value bytes]

Implementations can either store the value as-is (an external reader uses the same logical reassembly that Venice clients do — read first 4 bytes for the schema id, decompress the rest with the schema) or split out the schema id by parsing the prefix locally. Either way, a reader pulling from the external sink will see the same logical bytes a Venice client would observe after RocksDB's chunked-value reassembly: the chunking that Venice does for Kafka transport is invisible at this seam.

Byte arrays are passed through by reference for efficiency and must not be mutated by the implementation.

  • Field Details

    • SCHEMA_ID_PREFIX_LENGTH

      public static final int SCHEMA_ID_PREFIX_LENGTH
      Length of the 4-byte big-endian schema id prefix on every value.
      See Also:

  • Constructor Details

    • ExternalStorageRecord

      public ExternalStorageRecord(byte[] key, byte[] value)

  • Method Details

    • getKey

      public byte[] getKey()

    • getValue

      public byte[] getValue()
      Returns the value with the 4-byte big-endian schema id prefix in front of the Avro payload. The schema id can be parsed from the first SCHEMA_ID_PREFIX_LENGTH bytes.