Package com.linkedin.venice.utils

Class HelixUtils

java.lang.Object
com.linkedin.venice.utils.HelixUtils
public final class HelixUtils extends Object
Helper functions for Helix.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    DEFAULT_HELIX_OP_RETRY_COUNT
    Retry 9 times for each helix operation in case of getting the error by default.

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    checkClusterSetup(org.apache.helix.HelixAdmin admin, String cluster, int maxAttempts)
    Before this participant/spectator joining the Helix cluster, we need to make sure the cluster has been setup by controller.
    static <T> void
    compareAndUpdate(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, int maxAttempts, org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)
     
    static <T> void
    compareAndUpdate(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)
     
    static void
    connectHelixManager(SafeHelixManager manager, int maxAttempts)
    Try to connect Helix Manager.
    static <T> void
    create(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data)
     
    static <T> void
    create(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data, int maxAttempts)
     
    static <T> boolean
    createIfAbsent(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data, int maxAttempts, BiPredicate<T,T> sameData)
    Creates immutable data, or verifies that a concurrent creator persisted matching data.
    static <T> boolean
    exists(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path)
     
    static <T> List<T>
    getChildren(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, int maxAttempts)
     
    static org.apache.helix.model.CloudConfig
    getCloudConfig(org.apache.helix.cloud.constants.CloudProvider cloudProvider, String cloudId, List<String> cloudInfoSources, String cloudInfoProcessorPackage, String cloudInfoProcessorName)
     
    static org.apache.helix.model.MaintenanceSignal
    getClusterMaintenanceSignal(String clusterName, SafeHelixManager manager)
    If a cluster is in maintenance mode it returns all the signals associated with it regarding the reason and other details for creating the maintenance mode.
    static String
    getHelixClusterZkPath(String clusterName)
     
    static org.apache.helix.model.IdealState
    getIdealState(String clusterName, String resourceName, SafeHelixManager manager)
     
    static Instance
    getInstanceFromHelixInstanceName(String helixInstanceName)
     
    static int
    getPartitionId(String helixPartitionName)
     
    static String
    getPartitionName(String resourceName, int partitionId)
     
    static String
    getResourceName(String helixPartitionName)
     
    static String
    instanceIdToUrl(String instanceId)
    Convert a replica's instance ID to its URL.
    static String
    instanceIdToUrl(String instanceId, boolean https)
    Convert a replica's instance ID to its URL with a configurable scheme.
    static boolean
    isLiveInstance(String clusterName, String instanceId, SafeHelixManager manager)
     
    static <T> List<String>
    listPathContents(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path)
     
    static <T> void
    remove(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path)
     
    static <T> void
    remove(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, int maxAttempts)
     
    static void
    setupCustomizedStateConfig(org.apache.helix.HelixAdmin admin, String clusterName)
    This method tells helix what properties to aggregate for customized view.
    static <T> void
    update(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data)
     
    static <T> void
    update(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data, int maxAttempts)
     
    static <T> void
    updateChildren(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, List<String> paths, List<T> data)
     
    static <T> void
    updateChildren(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, List<String> paths, List<T> data, int maxAttempts)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • DEFAULT_HELIX_OP_RETRY_COUNT

      public static final int DEFAULT_HELIX_OP_RETRY_COUNT
      Retry 9 times for each helix operation in case of getting the error by default.
      See Also:

  • Method Details

    • getHelixClusterZkPath

      public static String getHelixClusterZkPath(String clusterName)

    • getPartitionId

      public static int getPartitionId(String helixPartitionName)

    • getPartitionName

      public static String getPartitionName(String resourceName, int partitionId)

    • getResourceName

      public static String getResourceName(String helixPartitionName)

    • getInstanceFromHelixInstanceName

      public static Instance getInstanceFromHelixInstanceName(String helixInstanceName)
      Parameters:
      helixInstanceName - of form host_port.
      Returns:
      Instance object with correct host and port.

    • getChildren

      public static <T> List<T> getChildren(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, int maxAttempts)
      Parameters:
      dataAccessor - ZK data accessor
      path - parent path
      maxAttempts - maximum number of retry attempts
      Returns:
      a list of objects that are under parent path. It will return an empty list of objects if parent path is not existing
      Throws:
      VeniceException - if data inconsistency persists after all retry attempts

    • listPathContents

      public static <T> List<String> listPathContents(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path)

    • create

      public static <T> void create(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data)

    • create

      public static <T> void create(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data, int maxAttempts)

    • createIfAbsent

      public static <T> boolean createIfAbsent(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data, int maxAttempts, BiPredicate<T,T> sameData)
      Creates immutable data, or verifies that a concurrent creator persisted matching data.
      Returns:
      true if this call created the node; false if matching data already existed
      Throws:
      VeniceException - if the node contains conflicting data

    • update

      public static <T> void update(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data)

    • update

      public static <T> void update(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, T data, int maxAttempts)

    • updateChildren

      public static <T> void updateChildren(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, List<String> paths, List<T> data)

    • updateChildren

      public static <T> void updateChildren(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, List<String> paths, List<T> data, int maxAttempts)

    • exists

      public static <T> boolean exists(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path)

    • remove

      public static <T> void remove(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path)

    • remove

      public static <T> void remove(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, int maxAttempts)

    • compareAndUpdate

      public static <T> void compareAndUpdate(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)

    • compareAndUpdate

      public static <T> void compareAndUpdate(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, String path, int maxAttempts, org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)

    • connectHelixManager

      public static void connectHelixManager(SafeHelixManager manager, int maxAttempts)
      Try to connect Helix Manager. If failed, waits for certain time and retry. If Helix Manager can not be connected after certain number of retry, throws exception. This method is most likely being used asynchronously since it is going to block and wait if connection fails.
      Parameters:
      manager - HelixManager instance
      maxAttempts - maximum number of attempts to retry on failure
      Throws:
      VeniceException - if connection keeps failing after certain number of retry

    • checkClusterSetup

      public static void checkClusterSetup(org.apache.helix.HelixAdmin admin, String cluster, int maxAttempts)
      Before this participant/spectator joining the Helix cluster, we need to make sure the cluster has been setup by controller. Otherwise, the following operations issued by participant/spectator would fail.

    • isLiveInstance

      public static boolean isLiveInstance(String clusterName, String instanceId, SafeHelixManager manager)

    • getIdealState

      public static org.apache.helix.model.IdealState getIdealState(String clusterName, String resourceName, SafeHelixManager manager)

    • getClusterMaintenanceSignal

      public static org.apache.helix.model.MaintenanceSignal getClusterMaintenanceSignal(String clusterName, SafeHelixManager manager)
      If a cluster is in maintenance mode it returns all the signals associated with it regarding the reason and other details for creating the maintenance mode.
      Parameters:
      clusterName -
      manager -
      Returns:
      the maintenance mode signal if the cluster is in maintenance mode, otherwise returns null.

    • setupCustomizedStateConfig

      public static void setupCustomizedStateConfig(org.apache.helix.HelixAdmin admin, String clusterName)
      This method tells helix what properties to aggregate for customized view. How customized view works in steps: 1. Venice SN reports its instance level states e.g. ingestion progress / hybrid store quota for each partition. 2. Helix aggregates the per instance level states to store version level. This method tells Helix what states to aggregate. So far we need to aggregate two properties HelixPartitionState.OFFLINE_PUSH and HelixPartitionState.HYBRID_STORE_QUOTA. 3. Other components e.g. Venice routers listen to the store version level states to serve traffic.
      Parameters:
      admin -
      clusterName -

    • instanceIdToUrl

      public static String instanceIdToUrl(String instanceId)
      Convert a replica's instance ID to its URL. An example is "localhost_port" to "https://localhost:port".
      Parameters:
      instanceId -
      Returns:
      replica URL

    • instanceIdToUrl

      public static String instanceIdToUrl(String instanceId, boolean https)
      Convert a replica's instance ID to its URL with a configurable scheme.
      Parameters:
      instanceId - the instance ID in the format "host_port"
      https - if true, use "https://"; otherwise, use "http://"
      Returns:
      replica URL

    • getCloudConfig

      public static org.apache.helix.model.CloudConfig getCloudConfig(org.apache.helix.cloud.constants.CloudProvider cloudProvider, String cloudId, List<String> cloudInfoSources, String cloudInfoProcessorPackage, String cloudInfoProcessorName)