Package com.linkedin.venice.pubsub

Class PubSubUtil

java.lang.Object
com.linkedin.venice.pubsub.PubSubUtil
public final class PubSubUtil extends Object

  • Constructor Details

    • PubSubUtil

      public PubSubUtil()

  • Method Details

    • getPubSubBrokerAddress

      public static String getPubSubBrokerAddress(Properties properties)

    • getPubSubBrokerAddress

      public static String getPubSubBrokerAddress(VeniceProperties properties)

    • getPubSubBrokerAddressWithDefault

      public static String getPubSubBrokerAddressWithDefault(VeniceProperties properties, String defaultValue)

    • getPubSubBrokerAddressOrFail

      public static String getPubSubBrokerAddressOrFail(VeniceProperties properties)

    • getPubSubBrokerAddressOrFail

      public static String getPubSubBrokerAddressOrFail(Properties properties)

    • addPubSubBrokerAddress

      public static Properties addPubSubBrokerAddress(Properties properties, String brokerAddress)

    • generatePubSubClientId

      public static String generatePubSubClientId(PubSubClientType pubSubClientType, String clientName, String brokerAddress)
      Generates a standardized and unique client ID for PubSub clients.

      This ensures uniqueness in client IDs, preventing naming collisions that could cause InstanceAlreadyExistsException during JMX metric registration. If multiple PubSub clients share the same client ID, underlying client libraries (e.g., Kafka) may fail JMX registration, resulting in runtime errors. By appending a timestamp, this method guarantees that each generated ID is unique.

      If the provided client name is null, it defaults to "venice-pubsub-client" followed by the host name. If the broker address is null, it defaults to an empty string. The generated client ID follows the format: 

      clientName-brokerAddress-timestamp

      Parameters:
      clientName - The name of the client (can be null, defaults to "venice-pubsub-client" + host name).
      brokerAddress - The PubSub broker address (can be null, defaults to an empty string).
      Returns:
      A unique client ID in the format: clientName-brokerAddress-timestamp.

    • getPubSubProducerConfigPrefix

      public static String getPubSubProducerConfigPrefix(String adapterConfigPrefix)

    • getPubSubConsumerConfigPrefix

      public static String getPubSubConsumerConfigPrefix(String adapterConfigPrefix)

    • getPubSubAdminConfigPrefix

      public static String getPubSubAdminConfigPrefix(String adapterConfigPrefix)

    • getPubSubSecurityProtocolOrDefault

      public static PubSubSecurityProtocol getPubSubSecurityProtocolOrDefault(VeniceProperties properties)
      TODO: Enforce explicit configuration of the PubSub security protocol in all components. Avoid defaulting to PubSubSecurityProtocol.PLAINTEXT. If the protocol is not explicitly defined via configuration, fail fast during startup to prevent silent misconfigurations.
      Parameters:
      properties - VeniceProperties containing configuration keys
      Returns:
      the resolved PubSubSecurityProtocol

    • getPubSubSecurityProtocolOrDefault

      public static PubSubSecurityProtocol getPubSubSecurityProtocolOrDefault(Properties properties)
      Returns the PubSubSecurityProtocol configured in the given Properties, falling back to PLAINTEXT if no value is found.
      Parameters:
      properties - the Java Properties object to extract the security protocol from
      Returns:
      the resolved PubSubSecurityProtocol, or PLAINTEXT if not specified

    • isPubSubSslProtocol

      public static boolean isPubSubSslProtocol(PubSubSecurityProtocol pubSubSecurityProtocol)
      Checks if the provided PubSubSecurityProtocol requires SSL.
      Parameters:
      pubSubSecurityProtocol - the security protocol to check
      Returns:
      true if the protocol uses SSL (either SSL or SASL_SSL), false otherwise

    • isPubSubSslProtocol

      public static boolean isPubSubSslProtocol(String pubSubSecurityProtocol)
      Checks if the given security protocol name corresponds to a protocol that requires SSL.
      Parameters:
      pubSubSecurityProtocol - the name of the security protocol (case-insensitive)
      Returns:
      true if the protocol uses SSL, false otherwise
      Throws:
      IllegalArgumentException - if the name does not correspond to a valid PubSubSecurityProtocol

    • computeOffsetDelta

      public static <T extends PubSubPosition> long computeOffsetDelta(PubSubTopicPartition partition, PubSubPosition position1, PubSubPosition position2, PubSubConsumerAdapter consumerAdapter)

    • calculateSeekOffset

      public static long calculateSeekOffset(long baseOffset, boolean isInclusive)
      Calculates the seek offset based on the base offset and inclusiveness flag.
      Parameters:
      baseOffset - the base offset to calculate from
      isInclusive - if true, returns the base offset; if false, returns base offset + 1
      Returns:
      the calculated seek offset

    • fromKafkaOffset

      public static PubSubPosition fromKafkaOffset(long offset)

    • getBase64EncodedString

      public static String getBase64EncodedString(byte[] byteBuffer)

    • getBase64DecodedBytes

      public static byte[] getBase64DecodedBytes(String bytesString)

    • parsePositionWireFormat

      public static PubSubPosition parsePositionWireFormat(String positionWireFormatString, PubSubPositionDeserializer pubSubPositionDeserializer)
      Parses a position wire format string and converts it to a PubSubPosition. The input string should be in the format "typeId:base64EncodedWfBytes".
      Parameters:
      positionWireFormatString - the position wire format string to parse
      pubSubPositionDeserializer - the deserializer to convert wire format to position
      Returns:
      the parsed PubSubPosition
      Throws:
      IllegalArgumentException - if the input string format is invalid

    • getPubSubPositionString

      public static String getPubSubPositionString(PubSubPositionDeserializer pubSubPositionDeserializer, ByteBuffer pubSubPosition)