Package com.linkedin.venice.utils

Class HelixUtils

  • public class HelixUtils
extends java.lang.Object
    Helper functions for Helix.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static int DEFAULT_HELIX_OP_RETRY_COUNT
      Retry 3 times for each helix operation in case of getting the error by default.
      static java.lang.String TOPOLOGY_CONSTRAINT
      The constraint that helix would apply on CRUSH alg.

    • Constructor Summary

      Constructors 
      Constructor Description
      HelixUtils()  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static void checkClusterSetup​(org.apache.helix.HelixAdmin admin, java.lang.String cluster, int maxRetry, int retryIntervalSec)
      Before this participant/spectator joining the Helix cluster, we need to make sure the cluster has been setup by controller.
      static <T> void compareAndUpdate​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, int retryCount, org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)  
      static <T> void compareAndUpdate​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)  
      static void connectHelixManager​(SafeHelixManager manager, int maxRetries, int sleepTimeSeconds)
      Try to connect Helix Manger.
      static <T> void create​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, T data)  
      static <T> void create​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, T data, int retryCount)  
      static <T> boolean exists​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path)  
      static <T> java.util.List<T> getChildren​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, int retryCount, long retryInterval)  
      static org.apache.helix.model.CloudConfig getCloudConfig​(org.apache.helix.cloud.constants.CloudProvider cloudProvider, java.lang.String cloudId, java.util.List<java.lang.String> cloudInfoSources, java.lang.String cloudInfoProcessorPackage, java.lang.String cloudInfoProcessorName)  
      static org.apache.helix.model.MaintenanceSignal getClusterMaintenanceSignal​(java.lang.String clusterName, SafeHelixManager manager)
      If a cluster is in maintenance mode it returns all the signals associated with it regarding the reason and other details for creating the maintenance mode.
      static java.lang.String getHelixClusterZkPath​(java.lang.String clusterName)  
      static Instance getInstanceFromHelixInstanceName​(java.lang.String helixInstanceName)  
      static int getPartitionId​(java.lang.String helixPartitionName)  
      static java.lang.String getPartitionName​(java.lang.String resourceName, int partitionId)  
      static java.lang.String getResourceName​(java.lang.String helixPartitionName)  
      static java.lang.String instanceIdToUrl​(java.lang.String instanceId)
      Convert a replica's instance ID to its URL.
      static boolean isLiveInstance​(java.lang.String clusterName, java.lang.String instanceId, SafeHelixManager manager)  
      static <T> java.util.List<java.lang.String> listPathContents​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path)  
      static <T> void remove​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path)  
      static <T> void remove​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, int retryCount)  
      static void setupCustomizedStateConfig​(org.apache.helix.HelixAdmin admin, java.lang.String clusterName)
      This method tells helix what properties to aggregate for customized view.
      static void setupInstanceConfig​(java.lang.String clusterName, java.lang.String instanceId, java.lang.String zkAddress)  
      static <T> void update​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, T data)  
      static <T> void update​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.lang.String path, T data, int retryCount)  
      static <T> void updateChildren​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.util.List<java.lang.String> paths, java.util.List<T> data)  
      static <T> void updateChildren​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor, java.util.List<java.lang.String> paths, java.util.List<T> data, int retryCount)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • DEFAULT_HELIX_OP_RETRY_COUNT

        public static final int DEFAULT_HELIX_OP_RETRY_COUNT
        Retry 3 times for each helix operation in case of getting the error by default.
        See Also:
        Constant Field Values

      • TOPOLOGY_CONSTRAINT

        public static final java.lang.String TOPOLOGY_CONSTRAINT
        The constraint that helix would apply on CRUSH alg. Based on this constraint, Helix would NOT allocate replicas in same partition to same instance. If we need rack aware ability in the future, we could add rack constraint as well.
        See Also:
        Constant Field Values

    • Constructor Detail

      • HelixUtils

        public HelixUtils()

    • Method Detail

      • getHelixClusterZkPath

        public static java.lang.String getHelixClusterZkPath​(java.lang.String clusterName)

      • getPartitionId

        public static int getPartitionId​(java.lang.String helixPartitionName)

      • getPartitionName

        public static java.lang.String getPartitionName​(java.lang.String resourceName,
                                                int partitionId)

      • getResourceName

        public static java.lang.String getResourceName​(java.lang.String helixPartitionName)

      • getInstanceFromHelixInstanceName

        public static Instance getInstanceFromHelixInstanceName​(java.lang.String helixInstanceName)
        Parameters:
        helixInstanceName - of form host_port.
        Returns:
        Instance object with correct host and port.

      • getChildren

        public static <T> java.util.List<T> getChildren​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                                                java.lang.String path,
                                                int retryCount,
                                                long retryInterval)
        Parameters:
        path - parent path
        Returns:
        a list of objects that are under parent path. It will return null if parent path is not existing

      • listPathContents

        public static <T> java.util.List<java.lang.String> listPathContents​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                                                                    java.lang.String path)

      • create

        public static <T> void create​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                              java.lang.String path,
                              T data)

      • create

        public static <T> void create​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                              java.lang.String path,
                              T data,
                              int retryCount)

      • update

        public static <T> void update​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                              java.lang.String path,
                              T data)

      • update

        public static <T> void update​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                              java.lang.String path,
                              T data,
                              int retryCount)

      • updateChildren

        public static <T> void updateChildren​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                                      java.util.List<java.lang.String> paths,
                                      java.util.List<T> data)

      • updateChildren

        public static <T> void updateChildren​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                                      java.util.List<java.lang.String> paths,
                                      java.util.List<T> data,
                                      int retryCount)

      • exists

        public static <T> boolean exists​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                                 java.lang.String path)

      • remove

        public static <T> void remove​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                              java.lang.String path)

      • remove

        public static <T> void remove​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                              java.lang.String path,
                              int retryCount)

      • compareAndUpdate

        public static <T> void compareAndUpdate​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                                        java.lang.String path,
                                        org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)

      • compareAndUpdate

        public static <T> void compareAndUpdate​(org.apache.helix.manager.zk.ZkBaseDataAccessor<T> dataAccessor,
                                        java.lang.String path,
                                        int retryCount,
                                        org.apache.helix.zookeeper.zkclient.DataUpdater<T> dataUpdater)

      • connectHelixManager

        public static void connectHelixManager​(SafeHelixManager manager,
                                       int maxRetries,
                                       int sleepTimeSeconds)
        Try to connect Helix Manger. If failed, waits for certain time and retry. If Helix Manager can not be connected after certain number of retry, throws exception. This method is most likely being used asynchronously since it is going to block and wait if connection fails.
        Parameters:
        manager - HelixManager instance
        maxRetries - retry time
        sleepTimeSeconds - time in second that it blocks until next retry.
        Throws:
        VeniceException - if connection keeps failing after certain number of retry

      • checkClusterSetup

        public static void checkClusterSetup​(org.apache.helix.HelixAdmin admin,
                                     java.lang.String cluster,
                                     int maxRetry,
                                     int retryIntervalSec)
        Before this participant/spectator joining the Helix cluster, we need to make sure the cluster has been setup by controller. Otherwise, the following operations issued by participant/spectator would fail.

      • setupInstanceConfig

        public static void setupInstanceConfig​(java.lang.String clusterName,
                                       java.lang.String instanceId,
                                       java.lang.String zkAddress)

      • isLiveInstance

        public static boolean isLiveInstance​(java.lang.String clusterName,
                                     java.lang.String instanceId,
                                     SafeHelixManager manager)

      • getClusterMaintenanceSignal

        public static org.apache.helix.model.MaintenanceSignal getClusterMaintenanceSignal​(java.lang.String clusterName,
                                                                                   SafeHelixManager manager)
        If a cluster is in maintenance mode it returns all the signals associated with it regarding the reason and other details for creating the maintenance mode.
        Parameters:
        clusterName -
        manager -
        Returns:
        the maintenance mode signal if the cluster is in maintenance mode, otherwise returns null.

      • setupCustomizedStateConfig

        public static void setupCustomizedStateConfig​(org.apache.helix.HelixAdmin admin,
                                              java.lang.String clusterName)
        This method tells helix what properties to aggregate for customized view. How customized view works in steps: 1. Venice SN reports its instance level states e.g. ingestion progress / hybrid store quota for each partition. 2. Helix aggregates the per instance level states to store version level. This method tells Helix what states to aggregate. So far we need to aggregate two properties HelixPartitionState.OFFLINE_PUSH and HelixPartitionState.HYBRID_STORE_QUOTA. 3. Other components e.g. Venice routers listen to the store version level states to serve traffic.
        Parameters:
        admin -
        clusterName -

      • instanceIdToUrl

        public static java.lang.String instanceIdToUrl​(java.lang.String instanceId)
        Convert a replica's instance ID to its URL. An example is "localhost_port" to "https://localhost:port".
        Parameters:
        instanceId -
        Returns:
        replica URL

      • getCloudConfig

        public static org.apache.helix.model.CloudConfig getCloudConfig​(org.apache.helix.cloud.constants.CloudProvider cloudProvider,
                                                                java.lang.String cloudId,
                                                                java.util.List<java.lang.String> cloudInfoSources,
                                                                java.lang.String cloudInfoProcessorPackage,
                                                                java.lang.String cloudInfoProcessorName)