Package com.linkedin.venice.fastclient

Class DelegatingAvroStoreClient<K,V>

java.lang.Object
com.linkedin.venice.fastclient.InternalAvroStoreClient<K,V>
com.linkedin.venice.fastclient.DelegatingAvroStoreClient<K,V>
All Implemented Interfaces:
AvroGenericReadComputeStoreClient<K,V>, AvroGenericStoreClient<K,V>, Closeable, AutoCloseable
Direct Known Subclasses:
DualReadAvroGenericStoreClient, LoadControlledAvroGenericStoreClient, RetriableAvroGenericStoreClient, StatsAvroGenericStoreClient
public class DelegatingAvroStoreClient<K,V> extends InternalAvroStoreClient<K,V>
Inside Fast-Client, we choose to use n-tier architecture style to build a pipeline to separate different types of logic in different layer.

n-tier architecture => Having multiple layers where each layer wraps the next inner one. Each layer provides some functionality, e.g. stats collection, etc.

Fast-Client's layers include the below components. Check ClientFactory.getAndStartGenericStoreClient(com.linkedin.venice.fastclient.ClientConfig) to figure out how the layers are put together for different requirements.

Layer -1:
AvroGenericStoreClient, AvroGenericReadComputeStoreClient => interfaces: Borrowed from thin-client

Layer 0:
InternalAvroStoreClient implements AvroGenericReadComputeStoreClient => The abstract class implementing above interfaces for fast-client. All other internal implementations of different tiers should extend this class.

Layer 1:
DispatchingAvroGenericStoreClient extends InternalAvroStoreClient => in charge of routing and serialization/de-serialization

Layer 2:
RetriableAvroGenericStoreClient extends DelegatingAvroStoreClient => Adds optional retry ability on top of DispatchingAvroGenericStoreClient

Layer 3:
StatsAvroGenericStoreClient extends DelegatingAvroStoreClient => Adds stats on top of Layer 2 or Layer 1. There is no option to disable it, but if needed, can be disabled.

Layer 4:
DualReadAvroGenericStoreClient extends DelegatingAvroStoreClient => Adds an extra read via thin client on top of Layer 3.

utils class:
DelegatingAvroStoreClient extends InternalAvroStoreClient => Delegator pattern to not override all the functions in every superclass in a duplicate manner.

Interactions between these class for some flows: https://swimlanes.io/u/iHTCBvlf0

  • Constructor Details

  • Method Details

    • getClientConfig

      public ClientConfig getClientConfig()
      Specified by:
      getClientConfig in class InternalAvroStoreClient<K,V>

    • getSchemaReader

      public SchemaReader getSchemaReader()

    • get

      protected CompletableFuture<V> get(GetRequestContext<K> requestContext, K key) throws VeniceClientException
      Specified by:
      get in class InternalAvroStoreClient<K,V>
      Throws:
      VeniceClientException

    • streamingBatchGet

      protected void streamingBatchGet(BatchGetRequestContext<K,V> requestContext, Set<K> keys, StreamingCallback<K,V> callback)
      Specified by:
      streamingBatchGet in class InternalAvroStoreClient<K,V>

    • compute

      protected void compute(ComputeRequestContext<K,V> requestContext, ComputeRequestWrapper computeRequestWrapper, Set<K> keys, org.apache.avro.Schema resultSchema, StreamingCallback<K,ComputeGenericRecord> callback, long preRequestTimeInNS) throws VeniceClientException
      Specified by:
      compute in class InternalAvroStoreClient<K,V>
      Throws:
      VeniceClientException

    • start

      public void start() throws VeniceClientException
      Throws:
      VeniceClientException

    • close

      public void close()
      Description copied from interface: AvroGenericStoreClient
      Release the internal resources.

    • getStoreName

      public String getStoreName()

    • getKeySchema

      public org.apache.avro.Schema getKeySchema()
      Description copied from interface: AvroGenericStoreClient
      Get key schema.

    • getLatestValueSchema

      @Deprecated public org.apache.avro.Schema getLatestValueSchema()
      Deprecated.
      Description copied from interface: AvroGenericStoreClient
      Get the latest value schema known in current store client. This function doesn't guarantee it will return the latest schema if you add a new value schema when current store client is running.

    • compute

      public ComputeRequestBuilder<K> compute(Optional<ClientStats> stats, Optional<ClientStats> streamingStats, AvroGenericReadComputeStoreClient computeStoreClient, long preRequestTimeInNS) throws VeniceClientException
      Throws:
      VeniceClientException

    • decompressAndDeserialize

      public V decompressAndDeserialize(ByteBuffer rawValue, int version, K key) throws VeniceClientException
      Description copied from interface: AvroGenericStoreClient
      Re-entry point for callers that obtained Venice-format raw value bytes from a path other than the standard Venice routing layer (e.g. an external storage system holding the same wire bytes) and want to run those bytes through the Fast Client's existing decompression + Avro deserialization pipeline without duplicating that logic.

      This entry point is Fast Client only — implemented by com.linkedin.venice.fastclient.DispatchingAvroGenericStoreClient and forwarded by Fast Client's DelegatingAvroStoreClient wrapper chain. Thin-client implementations inherit the throwing default because they lack the metadata refresh loop that supplies per-version compression state.

      Wire format. rawValue must hold the bytes exactly as Venice's storage layer writes them on disk / to an external sink: a 4-byte big-endian int writer-schema id followed by the post-compression Avro-serialized value body. This matches what com.linkedin.venice.writer.DualWriteVeniceWriter#toRocksDbFormattedValue produces for the dual-write external sink. The seam reads the schema id from those 4 bytes (so a store that has schema-evolved will still decode old payloads with their original writer schema), resolves the per-version compressor from StoreMetadata, decompresses the remainder, and deserializes with the embedded schema id.

      Parameters:
      rawValue - the value payload, prefixed by a 4-byte big-endian writer-schema id. Must not be null and must have at least 4 bytes.
      version - the store version the rawValue bytes were written under. Used to resolve the per-version compressor (including any ZSTD dictionary). Throws if the version is unknown to the metadata layer.
      key - the key that produced this value (used only for error messages and tracing)
      Returns:
      the deserialized value
      Throws:
      VeniceClientException - if the version is unknown, or if decompression or deserialization fails

    • registerVersionSwitchListener

      public void registerVersionSwitchListener(StoreVersionSwitchListener listener)
      Description copied from interface: AvroGenericStoreClient
      Register a StoreVersionSwitchListener that fires when the underlying metadata layer observes a change to the store's current serving version.

      Fast Client only. Thin-client implementations inherit the throwing default — there is no metadata refresh loop on the thin client, so version-switch transitions are not observable. See com.linkedin.venice.fastclient.DispatchingAvroGenericStoreClient for the Fast Client implementation and StoreVersionSwitchListener for threading and exception semantics.

      Registration timing. Listeners must be registered before AvroGenericStoreClient.start() to observe the initial transition committed by the first metadata refresh. Listeners registered after start() returns observe only subsequent transitions (and re-entrant registration from inside a callback observes only subsequent transitions as well). Callers needing the initial transition should use the non-starting factory entry point (com.linkedin.venice.fastclient.factory.ClientFactory#getGenericStoreClient), register their listeners, then call start() on the returned client.

      Registration is thread-safe.

    • registerStoreConfigChangeListener

      public void registerStoreConfigChangeListener(StoreConfigChangeListener listener)
      Description copied from interface: AvroGenericStoreClient
      Register a StoreConfigChangeListener that fires when the underlying metadata layer observes a change to the store-level config snapshot (e.g. operator-driven externalStorageReadMode flip).

      Fast Client only. Thin-client implementations inherit the throwing default. See com.linkedin.venice.fastclient.DispatchingAvroGenericStoreClient for the Fast Client implementation and StoreConfigChangeListener for threading and exception semantics.

      Registration timing. Same as AvroGenericStoreClient.registerVersionSwitchListener(com.linkedin.venice.client.store.listeners.StoreVersionSwitchListener) — register before AvroGenericStoreClient.start() to observe the initial snapshot; post-start registration observes only subsequent changes.