Package com.linkedin.venice.vpj

Class ExternalStorageWriteUtils

java.lang.Object
com.linkedin.venice.vpj.ExternalStorageWriteUtils
public final class ExternalStorageWriteUtils extends Object
Helpers for the VPJ external-storage dual-write path: the gating predicate, the typed reflective loader for ExternalStorageWriter impls, and the load+configure helper that ensures the impl is closed if configuration fails.

  • Method Details

    • isDualWriteToExternalStorageFromVpjEnabled

      public static boolean isDualWriteToExternalStorageFromVpjEnabled(VeniceProperties jobProps, StorageMode targetStorageMode)
      Returns true iff both halves of the dual-write gate are satisfied:
      • The VPJ-side push.job.external.storage.writer.class config is set to a non-empty value (i.e. an implementation is wired in for this push).
      • The store-version-side targetStorageMode is StorageMode.DUAL_WRITE (i.e. the version being pushed to has opted into dual-write at the store level).
      Either half missing means the path stays off and the partition writer falls back to today's Kafka-only produce.

    • loadExternalStorageWriter

      public static ExternalStorageWriter loadExternalStorageWriter(String className)
      Reflectively load and instantiate an ExternalStorageWriter implementation, validating that the configured class actually implements the interface before invoking its no-arg constructor.
      Throws:
      VeniceException - with a precise message when the class cannot be loaded, does not implement the SPI, or cannot be instantiated. Wrapping the underlying causes keeps stack traces useful while surfacing the configured class name in the message for operators triaging mis-configured pushes.

    • loadAndConfigure

      public static ExternalStorageWriter loadAndConfigure(String className, VeniceProperties jobProps, String topicName, int partitionId)
      Load the configured ExternalStorageWriter and invoke ExternalStorageWriter.configure(com.linkedin.venice.utils.VeniceProperties, java.lang.String, int). If configure throws, the partially-constructed writer is closed (best-effort) before the exception is propagated, so executor tasks do not leak connection pools / file handles / etc.