Package com.linkedin.venice.vpj

Class ExternalStorageWriteUtils

java.lang.Object
com.linkedin.venice.vpj.ExternalStorageWriteUtils
public final class ExternalStorageWriteUtils extends Object
Helpers for the VPJ external-storage dual-write path: the gating predicate, the typed reflective loader for ExternalStorageWriter impls, and the load+configure helper that ensures the impl is closed if configuration fails.

  • Method Details

    • isDualWriteToExternalStorageFromVpjEnabled

      public static boolean isDualWriteToExternalStorageFromVpjEnabled(VeniceProperties jobProps)
      Returns true iff both halves of the dual-write gate are satisfied:
      • The VPJ-side push.job.external.storage.writer.class config is set to a non-empty value (i.e. an implementation is wired in for this push).
      • The store-side push.job.dual.write.target.regions list is non-empty (i.e. at least one region's store-level storage mode is DUAL_WRITE). The VPJ driver resolves each region's storage mode and forwards only the DUAL_WRITE regions; an empty list means no region opted in.
      Either half missing means the path stays off and the partition writer falls back to today's Kafka-only produce.

    • getDualWriteTargetRegions

      public static List<String> getDualWriteTargetRegions(VeniceProperties jobProps)
      Parse the push.job.dual.write.target.regions config into the ordered list of region names whose store-level storage mode is DUAL_WRITE. Comma-separated; blank entries are dropped and duplicates are collapsed (first occurrence order preserved) so a repeated region never yields more than one writer. Returns an empty list when the key is absent or empty.

    • loadExternalStorageWriter

      public static ExternalStorageWriter loadExternalStorageWriter(String className)
      Reflectively load and instantiate an ExternalStorageWriter implementation, validating that the configured class actually implements the interface before invoking its no-arg constructor.
      Throws:
      VeniceException - with a precise message when the class cannot be loaded, does not implement the SPI, or cannot be instantiated. Wrapping the underlying causes keeps stack traces useful while surfacing the configured class name in the message for operators triaging mis-configured pushes.

    • loadAndConfigure

      public static ExternalStorageWriter loadAndConfigure(String className, VeniceProperties jobProps, String topicName, int partitionId, String region)
      Load the configured ExternalStorageWriter and invoke ExternalStorageWriter.configure(VeniceProperties, String, int, String) for region. If configure throws, the partially-constructed writer is closed (best-effort) before the exception is propagated, so executor tasks do not leak connection pools / file handles / etc.