Package com.linkedin.davinci.blobtransfer

Class BlobTransferUtils

java.lang.Object
com.linkedin.davinci.blobtransfer.BlobTransferUtils
public class BlobTransferUtils extends Object

  • Field Details

    • BLOB_TRANSFER_STATUS

      public static final String BLOB_TRANSFER_STATUS
      See Also:

    • BLOB_TRANSFER_COMPLETED

      public static final String BLOB_TRANSFER_COMPLETED
      See Also:

    • BLOB_TRANSFER_TYPE

      public static final String BLOB_TRANSFER_TYPE
      See Also:

    • BLOB_TRANSFER_PARTITION_STATE_SCHEMA_VERSION

      public static final String BLOB_TRANSFER_PARTITION_STATE_SCHEMA_VERSION
      Protocol version of the PartitionState schema the requester is compiled against. Set by the client on the blob-transfer request so the server can fail fast with a 412 PRECONDITION_FAILED before any file work begins.
      See Also:

    • BLOB_TRANSFER_STORE_VERSION_STATE_SCHEMA_VERSION

      public static final String BLOB_TRANSFER_STORE_VERSION_STATE_SCHEMA_VERSION
      Same purpose as BLOB_TRANSFER_PARTITION_STATE_SCHEMA_VERSION but for StoreVersionState.
      See Also:

    • BLOB_TRANSFER_REQUEST_ORIGIN

      public static final io.netty.util.AttributeKey<BlobTransferUtils.BlobTransferRequestOrigin> BLOB_TRANSFER_REQUEST_ORIGIN
      Channel attribute carrying the BlobTransferUtils.BlobTransferRequestOrigin of an inbound request. Set by BlobTransferAclHandler during the TLS/ACL stage and read by downstream handlers (P2PFileTransferServerHandler, admission control) for prioritization and store-level ACL.

  • Constructor Details

    • BlobTransferUtils

      public BlobTransferUtils()

  • Method Details

    • isMetadataMessage

      public static boolean isMetadataMessage(io.netty.handler.codec.http.HttpResponse msg)
      Check if the HttpResponse message is for metadata.
      Parameters:
      msg - the HttpResponse message
      Returns:
      true if the message is a metadata message, false otherwise

    • compareRequestedSchemaVersionsAgainstLocal

      public static String compareRequestedSchemaVersionsAgainstLocal(io.netty.handler.codec.http.HttpRequest request)
      Compare the schema-version headers on a P2P blob-transfer GET request against the local binary's compiled-in currentProtocolVersion. Used by the server right next to the table-format check so a schema mismatch is rejected with a 412 PRECONDITION_FAILED before any file work begins — otherwise the client would pay for the entire file transfer and only discover the mismatch at the metadata stage.

      An exact-match policy is used (rather than e.g. "peer <= local"). Blob transfer is the fast path; if the binaries on the two ends are not in lock-step we want to step aside and let Kafka bootstrap take over rather than rely on cross-version Avro promotion of the partition metadata. Skew between peers is limited to rolling-deploy windows, so the cost of being strict is bounded.

      Behaviour for the absent / malformed cases is intentionally permissive so a server-side rollout of the new headers cannot break peers that haven't been upgraded yet, and so a header parsing bug cannot reject a request:

      • Both headers absent — pass through (peer is on an older binary).
      • Header value non-numeric or out of byte range — pass through; the existing deserialization-time exception remains as the safety net for the truly incompatible case (no regression vs. today).

      Returns a diagnostic string suitable for the response body when the request is incompatible, or null when it is compatible.

    • generateFileChecksum

      public static String generateFileChecksum(Path filePath) throws IOException
      Generate MD5 checksum for a file
      Parameters:
      filePath - the path to the file
      Returns:
      a hex string
      Throws:
      IOException - if an I/O error occurs

    • createSSLFactoryForBlobTransferInDVC

      public static Optional<SSLFactory> createSSLFactoryForBlobTransferInDVC(VeniceConfigLoader configLoader)
      Create an SSLFactory from the Venice config loader
      Parameters:
      configLoader - The Venice config loader containing SSL configuration
      Returns:
      Optional SSLFactory, which will be empty if SSL is not enabled

    • createBlobTransferClientSslHandler

      public static io.netty.handler.ssl.SslHandler createBlobTransferClientSslHandler(Optional<SSLFactory> sslFactory)

    • createAclHandler

      public static Optional<BlobTransferAclHandler> createAclHandler(VeniceConfigLoader configLoader, Optional<DynamicAccessController> storeAccessController, boolean serverAcceptClientBlobRequestEnabled)
      Create the acl handler for blob transfer, for both DVC peers and server peers.
      Parameters:
      serverAcceptClientBlobRequestEnabled - whether this server accepts client-origin requests; gates the client-origin accept check and per-store read ACL in the returned handler.

    • isBlobTransferManagerEnabled

      public static boolean isBlobTransferManagerEnabled(VeniceServerConfig backendConfig)
      A config check to determine if blob transfer manager is enabled
      Parameters:
      backendConfig - the Venice server config
      Returns:
      true if blob transfer manager is enabled, false otherwise