Package com.linkedin.venice.controller

Interface Admin

  • All Superinterfaces:
    java.lang.AutoCloseable, java.io.Closeable
    All Known Implementing Classes:
    VeniceHelixAdmin, VeniceParentHelixAdmin
    public interface Admin
extends java.lang.AutoCloseable, java.io.Closeable

    • Method Detail

      • initStorageCluster

        void initStorageCluster​(java.lang.String clusterName)

      • isClusterValid

        boolean isClusterValid​(java.lang.String clusterName)

      • createStore

        default void createStore​(java.lang.String clusterName,
                         java.lang.String storeName,
                         java.lang.String owner,
                         java.lang.String keySchema,
                         java.lang.String valueSchema)

      • createStore

        default void createStore​(java.lang.String clusterName,
                         java.lang.String storeName,
                         java.lang.String owner,
                         java.lang.String keySchema,
                         java.lang.String valueSchema,
                         boolean isSystemStore)

      • createStore

        void createStore​(java.lang.String clusterName,
                 java.lang.String storeName,
                 java.lang.String owner,
                 java.lang.String keySchema,
                 java.lang.String valueSchema,
                 boolean isSystemStore,
                 java.util.Optional<java.lang.String> accessPermissions)

      • isStoreMigrationAllowed

        boolean isStoreMigrationAllowed​(java.lang.String srcClusterName)

      • migrateStore

        void migrateStore​(java.lang.String srcClusterName,
                  java.lang.String destClusterName,
                  java.lang.String storeName)

      • completeMigration

        void completeMigration​(java.lang.String srcClusterName,
                       java.lang.String destClusterName,
                       java.lang.String storeName)

      • abortMigration

        void abortMigration​(java.lang.String srcClusterName,
                    java.lang.String destClusterName,
                    java.lang.String storeName)

      • deleteStore

        void deleteStore​(java.lang.String clusterName,
                 java.lang.String storeName,
                 int largestUsedVersionNumber,
                 boolean waitOnRTTopicDeletion)
        Delete the entire store including both metadata and real user's data. Before deleting a store, we should disable the store manually to ensure there is no reading/writing request hitting this tore.

      • addVersionAndStartIngestion

        void addVersionAndStartIngestion​(java.lang.String clusterName,
                                 java.lang.String storeName,
                                 java.lang.String pushJobId,
                                 int versionNumber,
                                 int numberOfPartitions,
                                 Version.PushType pushType,
                                 java.lang.String remoteKafkaBootstrapServers,
                                 long rewindTimeInSecondsOverride,
                                 int replicationMetadataVersionId,
                                 boolean versionSwapDeferred,
                                 int repushSourceVersion)
        This method behaves differently in VeniceHelixAdmin and VeniceParentHelixAdmin.

      • hasWritePermissionToBatchJobHeartbeatStore

        default boolean hasWritePermissionToBatchJobHeartbeatStore​(java.security.cert.X509Certificate requesterCert,
                                                           java.lang.String batchJobHeartbeatStoreName)
                                                    throws AclException
        Throws:
        AclException

      • incrementVersionIdempotent

        default Version incrementVersionIdempotent​(java.lang.String clusterName,
                                           java.lang.String storeName,
                                           java.lang.String pushJobId,
                                           int numberOfPartitions,
                                           int replicationFactor)
        The implementation of this method must take no action and return the same Version object if the same parameters are provided on a subsequent invocation. The expected use is multiple distributed components of a single push (with a single jobPushId) that each need to query Venice for the Version (and Kafka topic) to write into. The first task triggers a new Version, all subsequent tasks identify with the same jobPushId, and should be provided with the same Version object.

      • incrementVersionIdempotent

        default Version incrementVersionIdempotent​(java.lang.String clusterName,
                                           java.lang.String storeName,
                                           java.lang.String pushJobId,
                                           int numberOfPartitions,
                                           int replicationFactor,
                                           Version.PushType pushType,
                                           boolean sendStartOfPush,
                                           boolean sorted,
                                           java.lang.String compressionDictionary,
                                           java.util.Optional<java.lang.String> sourceGridFabric,
                                           java.util.Optional<java.security.cert.X509Certificate> requesterCert,
                                           long rewindTimeInSecondsOverride,
                                           java.util.Optional<java.lang.String> emergencySourceRegion,
                                           boolean versionSwapDeferred,
                                           int repushSourceVersion)

      • incrementVersionIdempotent

        Version incrementVersionIdempotent​(java.lang.String clusterName,
                                   java.lang.String storeName,
                                   java.lang.String pushJobId,
                                   int numberOfPartitions,
                                   int replicationFactor,
                                   Version.PushType pushType,
                                   boolean sendStartOfPush,
                                   boolean sorted,
                                   java.lang.String compressionDictionary,
                                   java.util.Optional<java.lang.String> sourceGridFabric,
                                   java.util.Optional<java.security.cert.X509Certificate> requesterCert,
                                   long rewindTimeInSecondsOverride,
                                   java.util.Optional<java.lang.String> emergencySourceRegion,
                                   boolean versionSwapDeferred,
                                   java.lang.String targetedRegions,
                                   int repushSourceVersion)

      • getRealTimeTopic

        java.lang.String getRealTimeTopic​(java.lang.String clusterName,
                                  Store store)

      • getRealTimeTopic

        default java.lang.String getRealTimeTopic​(java.lang.String clusterName,
                                          java.lang.String storeName)

      • getSeparateRealTimeTopic

        java.lang.String getSeparateRealTimeTopic​(java.lang.String clusterName,
                                          java.lang.String storeName)

      • getIncrementalPushVersion

        Version getIncrementalPushVersion​(java.lang.String clusterName,
                                  java.lang.String storeName)
        Right now, it will return the latest version recorded in parent controller. There are a couple of edge cases. 1. If a push fails in some colos, the version will be inconsistent among colos 2. If rollback happens, latest version will not be the current version. TODO: figure out how we'd like to cover these edge cases

      • getCurrentVersion

        int getCurrentVersion​(java.lang.String clusterName,
                      java.lang.String storeName)

      • getCurrentVersionsForMultiColos

        java.util.Map<java.lang.String,​java.lang.Integer> getCurrentVersionsForMultiColos​(java.lang.String clusterName,
                                                                                        java.lang.String storeName)

      • getFutureVersionsForMultiColos

        java.util.Map<java.lang.String,​java.lang.String> getFutureVersionsForMultiColos​(java.lang.String clusterName,
                                                                                      java.lang.String storeName)

      • getBackupVersionsForMultiColos

        java.util.Map<java.lang.String,​java.lang.String> getBackupVersionsForMultiColos​(java.lang.String clusterName,
                                                                                      java.lang.String storeName)

      • getBackupVersion

        int getBackupVersion​(java.lang.String clusterName,
                     java.lang.String storeName)

      • getFutureVersion

        int getFutureVersion​(java.lang.String clusterName,
                     java.lang.String storeName)

      • getRepushInfo

        RepushInfo getRepushInfo​(java.lang.String clusterNae,
                         java.lang.String storeName,
                         java.util.Optional<java.lang.String> fabricName)

      • peekNextVersion

        Version peekNextVersion​(java.lang.String clusterName,
                        java.lang.String storeName)

      • deleteAllVersionsInStore

        java.util.List<Version> deleteAllVersionsInStore​(java.lang.String clusterName,
                                                 java.lang.String storeName)
        Delete all of venice versions in given store(including venice resource, kafka topic, offline pushs and all related resources).
        Throws:
        VeniceException - If the given store was not disabled, an exception would be thrown to reject deletion request.

      • deleteOldVersionInStore

        void deleteOldVersionInStore​(java.lang.String clusterName,
                             java.lang.String storeName,
                             int versionNum)
        Delete the given version from the store. If the given version is the current version, an exception will be thrown.

      • versionsForStore

        java.util.List<Version> versionsForStore​(java.lang.String clusterName,
                                         java.lang.String storeName)

      • getAllStores

        java.util.List<Store> getAllStores​(java.lang.String clusterName)

      • getAllStoreStatuses

        java.util.Map<java.lang.String,​java.lang.String> getAllStoreStatuses​(java.lang.String clusterName)
        Get the statuses of all stores. The store status is decided by the current version. For example, if one partition only have 2 ONLINE replicas in the current version, we say this store is under replicated. Refer to StoreStatus for the definition of each status.
        Returns:
        a map whose key is store name and value is store's status.

      • getStore

        Store getStore​(java.lang.String clusterName,
               java.lang.String storeName)

      • hasStore

        boolean hasStore​(java.lang.String clusterName,
                 java.lang.String storeName)

      • getKeySchema

        SchemaEntry getKeySchema​(java.lang.String clusterName,
                         java.lang.String storeName)

      • getValueSchemas

        java.util.Collection<SchemaEntry> getValueSchemas​(java.lang.String clusterName,
                                                  java.lang.String storeName)

      • getDerivedSchemas

        java.util.Collection<DerivedSchemaEntry> getDerivedSchemas​(java.lang.String clusterName,
                                                           java.lang.String storeName)

      • getValueSchemaId

        int getValueSchemaId​(java.lang.String clusterName,
                     java.lang.String storeName,
                     java.lang.String valueSchemaStr)

      • getDerivedSchemaId

        GeneratedSchemaID getDerivedSchemaId​(java.lang.String clusterName,
                                     java.lang.String storeName,
                                     java.lang.String schemaStr)

      • getValueSchema

        SchemaEntry getValueSchema​(java.lang.String clusterName,
                           java.lang.String storeName,
                           int id)

      • addSupersetSchema

        SchemaEntry addSupersetSchema​(java.lang.String clusterName,
                              java.lang.String storeName,
                              java.lang.String valueSchemaStr,
                              int valueSchemaId,
                              java.lang.String supersetSchemaStr,
                              int supersetSchemaId)

      • addDerivedSchema

        DerivedSchemaEntry addDerivedSchema​(java.lang.String clusterName,
                                    java.lang.String storeName,
                                    int valueSchemaId,
                                    java.lang.String derivedSchemaStr)

      • getInUseValueSchemaIds

        java.util.Set<java.lang.Integer> getInUseValueSchemaIds​(java.lang.String clusterName,
                                                        java.lang.String storeName)

      • deleteValueSchemas

        void deleteValueSchemas​(java.lang.String clusterName,
                        java.lang.String storeName,
                        java.util.Set<java.lang.Integer> inuseValueSchemaIds)
        Deletes a store's values schema with ids `except` the ids passed in the argument inuseValueSchemaIds

      • addDerivedSchema

        DerivedSchemaEntry addDerivedSchema​(java.lang.String clusterName,
                                    java.lang.String storeName,
                                    int valueSchemaId,
                                    int derivedSchemaId,
                                    java.lang.String derivedSchemaStr)
        This method skips most precondition checks and is intended for only internal use.

      • getReplicationMetadataSchemas

        java.util.Collection<RmdSchemaEntry> getReplicationMetadataSchemas​(java.lang.String clusterName,
                                                                   java.lang.String storeName)

      • getReplicationMetadataSchema

        java.util.Optional<org.apache.avro.Schema> getReplicationMetadataSchema​(java.lang.String clusterName,
                                                                        java.lang.String storeName,
                                                                        int valueSchemaID,
                                                                        int rmdVersionID)

      • addReplicationMetadataSchema

        RmdSchemaEntry addReplicationMetadataSchema​(java.lang.String clusterName,
                                            java.lang.String storeName,
                                            int valueSchemaId,
                                            int replicationMetadataVersionId,
                                            java.lang.String replicationMetadataSchemaStr)

      • validateAndMaybeRetrySystemStoreAutoCreation

        void validateAndMaybeRetrySystemStoreAutoCreation​(java.lang.String clusterName,
                                                  java.lang.String storeName,
                                                  VeniceSystemStoreType systemStoreType)

      • removeDerivedSchema

        DerivedSchemaEntry removeDerivedSchema​(java.lang.String clusterName,
                                       java.lang.String storeName,
                                       int valueSchemaId,
                                       int derivedSchemaId)
        Remove an existing derived schema
        Returns:
        the derived schema that is deleted or null if the schema doesn't exist

      • setStoreCurrentVersion

        void setStoreCurrentVersion​(java.lang.String clusterName,
                            java.lang.String storeName,
                            int versionNumber)

      • rollForwardToFutureVersion

        void rollForwardToFutureVersion​(java.lang.String clusterName,
                                java.lang.String storeName,
                                java.lang.String regionFilter)

      • rollbackToBackupVersion

        void rollbackToBackupVersion​(java.lang.String clusterName,
                             java.lang.String storeName,
                             java.lang.String regionFilter)

      • setStoreLargestUsedVersion

        void setStoreLargestUsedVersion​(java.lang.String clusterName,
                                java.lang.String storeName,
                                int versionNumber)

      • setStoreOwner

        void setStoreOwner​(java.lang.String clusterName,
                   java.lang.String storeName,
                   java.lang.String owner)

      • setStorePartitionCount

        void setStorePartitionCount​(java.lang.String clusterName,
                            java.lang.String storeName,
                            int partitionCount)

      • setStoreReadability

        void setStoreReadability​(java.lang.String clusterName,
                         java.lang.String storeName,
                         boolean desiredReadability)

      • setStoreWriteability

        void setStoreWriteability​(java.lang.String clusterName,
                          java.lang.String storeName,
                          boolean desiredWriteability)

      • setStoreReadWriteability

        void setStoreReadWriteability​(java.lang.String clusterName,
                              java.lang.String storeName,
                              boolean isAccessible)

      • updateStore

        void updateStore​(java.lang.String clusterName,
                 java.lang.String storeName,
                 UpdateStoreQueryParams params)

      • getStorageEngineOverheadRatio

        double getStorageEngineOverheadRatio​(java.lang.String clusterName)

      • getStorageNodes

        java.util.List<java.lang.String> getStorageNodes​(java.lang.String clusterName)

      • getStorageNodesStatus

        java.util.Map<java.lang.String,​java.lang.String> getStorageNodesStatus​(java.lang.String clusterName,
                                                                             boolean enableReplica)

      • removeStorageNode

        void removeStorageNode​(java.lang.String clusterName,
                       java.lang.String instanceId)

      • stop

        void stop​(java.lang.String clusterName)
        Stop the helix controller for a single cluster.

      • stopVeniceController

        void stopVeniceController()
        Stop the entire controller but not only the helix controller for a single cluster.

      • getOffLinePushStatus

        Admin.OfflinePushStatusInfo getOffLinePushStatus​(java.lang.String clusterName,
                                                 java.lang.String kafkaTopic)
        Query the status of the offline push by given kafka topic. TODO We use kafka topic to tracking the status now but in the further we should use jobId instead of kafka TODO topic. Right now each kafka topic only have one offline job. But in the further one kafka topic could be TODO assigned multiple jobs like data migration job etc.
        Returns:
        the status of current offline push for the passed kafka topic

      • getOffLinePushStatus

        Admin.OfflinePushStatusInfo getOffLinePushStatus​(java.lang.String clusterName,
                                                 java.lang.String kafkaTopic,
                                                 java.util.Optional<java.lang.String> incrementalPushVersion,
                                                 java.lang.String region,
                                                 java.lang.String targetedRegions)

      • getKafkaBootstrapServers

        java.lang.String getKafkaBootstrapServers​(boolean isSSL)
        Return the ssl or non-ssl bootstrap servers based on the given flag.
        Returns:
        kafka bootstrap servers url, if there are multiple will be comma separated.

      • getRegionName

        java.lang.String getRegionName()
        Return the region name of this Admin
        Returns:
        the region name of this controller

      • getNativeReplicationKafkaBootstrapServerAddress

        java.lang.String getNativeReplicationKafkaBootstrapServerAddress​(java.lang.String sourceFabric)

      • getNativeReplicationSourceFabric

        java.lang.String getNativeReplicationSourceFabric​(java.lang.String clusterName,
                                                  Store store,
                                                  java.util.Optional<java.lang.String> sourceGridFabric,
                                                  java.util.Optional<java.lang.String> emergencySourceRegion,
                                                  java.lang.String targetedRegions)

      • isSSLEnabledForPush

        boolean isSSLEnabledForPush​(java.lang.String clusterName,
                            java.lang.String storeName)
        Return whether ssl is enabled for the given store for push.

      • isSslToKafka

        boolean isSslToKafka()

      • getTopicManager

        TopicManager getTopicManager​(java.lang.String pubSubServerAddress)

      • getAggregatedHealthStatus

        InstanceRemovableStatuses getAggregatedHealthStatus​(java.lang.String cluster,
                                                    java.util.List<java.lang.String> instances,
                                                    java.util.List<java.lang.String> toBeStoppedInstances,
                                                    boolean isSSLEnabled)

      • isRTTopicDeletionPermittedByAllControllers

        boolean isRTTopicDeletionPermittedByAllControllers​(java.lang.String clusterName,
                                                   java.lang.String storeName)

      • isLeaderControllerFor

        boolean isLeaderControllerFor​(java.lang.String clusterName)
        Check if this controller itself is the leader controller for a given cluster or not. Note that the controller can be either a parent controller or a child controller since a cluster must have a leader child controller and a leader parent controller. The point is not to be confused the concept of leader-standby with parent-child controller architecture.

      • calculateNumberOfPartitions

        int calculateNumberOfPartitions​(java.lang.String clusterName,
                                java.lang.String storeName)
        Calculate how many partitions are needed for the given store.

      • getReplicationFactor

        int getReplicationFactor​(java.lang.String clusterName,
                         java.lang.String storeName)

      • getDatacenterCount

        default int getDatacenterCount​(java.lang.String clusterName)
        number of datacenters, 1 if in single cluster mode. Could be more if this is a parent controller

      • getReplicas

        java.util.List<Replica> getReplicas​(java.lang.String clusterName,
                                    java.lang.String kafkaTopic)

      • getReplicasOfStorageNode

        java.util.List<Replica> getReplicasOfStorageNode​(java.lang.String clusterName,
                                                 java.lang.String instanceId)

      • isInstanceRemovable

        NodeRemovableResult isInstanceRemovable​(java.lang.String clusterName,
                                        java.lang.String helixNodeId,
                                        java.util.List<java.lang.String> lockedNodes)
        Assuming all hosts identified by lockedNodes and their corresponding resources are unusable, is the given instance able to be removed out from the given cluster. For example, if there is only one online replica alive in this cluster which is hosted on the given instance. This instance should not be removed out of cluster, otherwise Venice will lose data. For detail criteria please refer to InstanceStatusDecider
        Parameters:
        clusterName - The cluster were the hosts belong.
        helixNodeId - nodeId of helix participant. HOST_PORT.
        lockedNodes - A list of helix nodeIds whose resources are assumed to be unusable (stopped).

      • getLeaderController

        Instance getLeaderController​(java.lang.String clusterName)
        Get instance of leader controller. If there is no leader controller for the given cluster, throw a VeniceException.

      • addInstanceToAllowlist

        void addInstanceToAllowlist​(java.lang.String clusterName,
                            java.lang.String helixNodeId)

      • removeInstanceFromAllowList

        void removeInstanceFromAllowList​(java.lang.String clusterName,
                                 java.lang.String helixNodeId)

      • getAllowlist

        java.util.Set<java.lang.String> getAllowlist​(java.lang.String clusterName)

      • killOfflinePush

        void killOfflinePush​(java.lang.String clusterName,
                     java.lang.String kafkaTopic,
                     boolean isForcedKill)
        Kill an offline push if it ran into errors or the corresponding version is being retired.
        Parameters:
        clusterName -
        kafkaTopic -
        isForcedKill - should be set to true when killing the push job for retiring the corresponding version.

      • getStorageNodesStatus

        StorageNodeStatus getStorageNodesStatus​(java.lang.String clusterName,
                                        java.lang.String instanceId)
        Query and return the current status of the given storage node. The "storage node status" is composed by "status" of all replicas in that storage node. "status" is an integer value of Helix state:
        • DROPPED=1
        • ERROR=2
        • OFFLINE=3
        • BOOTSTRAP=4
        • ONLINE=5
        So this method will return a map, the key is the replica name which is composed by resource name and partitionId, and the value is the "status" of this replica.

      • isStorageNodeNewerOrEqualTo

        boolean isStorageNodeNewerOrEqualTo​(java.lang.String clusterName,
                                    java.lang.String instanceId,
                                    StorageNodeStatus oldServerStatus)
        Compare the current storage node status and the given storage node status to check is the current one is "Newer" or "Equal" to the given one. Compare will go through each of replica in this storage node, if all their statuses values were larger or equal than the statuses value in the given storage node status, We say current storage node status is "Newer" or "Equal " to the given one.

      • setAdminConsumerService

        void setAdminConsumerService​(java.lang.String clusterName,
                             AdminConsumerService service)

      • skipAdminMessage

        void skipAdminMessage​(java.lang.String clusterName,
                      long offset,
                      boolean skipDIV)
        The admin consumption task tries to deal with failures to process an admin message by retrying. If there is a message that cannot be processed for some reason, we will need to forcibly skip that message in order to unblock the task from consuming subsequent messages.
        Parameters:
        clusterName -
        offset -
        skipDIV - tries to skip only the DIV check for the blocking message.

      • getLastSucceedExecutionId

        java.lang.Long getLastSucceedExecutionId​(java.lang.String clusterName)
        Get the id of the last succeed execution in this controller.

      • getAdminCommandExecutionTracker

        java.util.Optional<AdminCommandExecutionTracker> getAdminCommandExecutionTracker​(java.lang.String clusterName)
        Get the tracker used to track the execution of the admin command for the given cluster.

      • getRoutersClusterConfig

        RoutersClusterConfig getRoutersClusterConfig​(java.lang.String clusterName)
        Get the cluster level config for all routers.

      • updateRoutersClusterConfig

        void updateRoutersClusterConfig​(java.lang.String clusterName,
                                java.util.Optional<java.lang.Boolean> isThrottlingEnable,
                                java.util.Optional<java.lang.Boolean> isQuotaRebalancedEnable,
                                java.util.Optional<java.lang.Boolean> isMaxCapaictyProtectionEnabled,
                                java.util.Optional<java.lang.Integer> expectedRouterCount)
        Update the cluster level for all routers.

      • getAllStorePushStrategyForMigration

        java.util.Map<java.lang.String,​java.lang.String> getAllStorePushStrategyForMigration()

      • setStorePushStrategyForMigration

        void setStorePushStrategyForMigration​(java.lang.String voldemortStoreName,
                                      java.lang.String strategy)

      • discoverCluster

        Pair<java.lang.String,​java.lang.String> discoverCluster​(java.lang.String storeName)
        Find the cluster which the given store belongs to. Return the pair of the cluster name and the d2 service associated with that cluster.
        Throws:
        VeniceException - if not cluster is found.

      • getServerD2Service

        java.lang.String getServerD2Service​(java.lang.String clusterName)
        Find the server d2 service associated with a given cluster name.

      • findAllBootstrappingVersions

        java.util.Map<java.lang.String,​java.lang.String> findAllBootstrappingVersions​(java.lang.String clusterName)
        Find the store versions which have at least one bootstrap replica.

      • getPubSubSSLProperties

        VeniceProperties getPubSubSSLProperties​(java.lang.String pubSubBrokerAddress)

      • close

        void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable

      • isLeaderControllerOfControllerCluster

        boolean isLeaderControllerOfControllerCluster()
        This function can be used to perform cluster-wide operations which need to be performed by a single process only in the whole cluster. There could be a race condition during leader controller failover, and so long operation should have some way of guarding against that.

      • isTopicTruncated

        boolean isTopicTruncated​(java.lang.String topicName)

      • isTopicTruncatedBasedOnRetention

        boolean isTopicTruncatedBasedOnRetention​(long retention)

      • isTopicTruncatedBasedOnRetention

        boolean isTopicTruncatedBasedOnRetention​(java.lang.String topicName,
                                         long retention)

      • getMinNumberOfUnusedKafkaTopicsToPreserve

        int getMinNumberOfUnusedKafkaTopicsToPreserve()

      • truncateKafkaTopic

        boolean truncateKafkaTopic​(java.lang.String topicName)
        Returns:
        false indicates that the truncate operation has already been done before; true if it's the first time truncating this topic.

      • truncateKafkaTopic

        boolean truncateKafkaTopic​(java.lang.String topicName,
                           long retentionTimeInMs)
        Truncate a Kafka topic by setting its retention time to the input value.
        Parameters:
        topicName - the name of the topic to truncate.
        retentionTimeInMs - the retention time in milliseconds to set for the topic.
        Returns:
        true if truncating this topic successfully. false otherwise.

      • isResourceStillAlive

        boolean isResourceStillAlive​(java.lang.String resourceName)
        Check whether the specified resource is fully removed or not.

      • updateClusterDiscovery

        void updateClusterDiscovery​(java.lang.String storeName,
                            java.lang.String oldCluster,
                            java.lang.String newCluster,
                            java.lang.String initiatingCluster)
        Update the cluster discovery of a given store by writing to the StoreConfig ZNode.
        Parameters:
        storeName - of the store.
        oldCluster - for the store.
        newCluster - for the store.
        initiatingCluster - that is making the update. This is needed because in the case of store migration sometimes the update is not made by the leader of the current cluster but instead the leader of the source cluster.

      • writeEndOfPush

        void writeEndOfPush​(java.lang.String clusterName,
                    java.lang.String storeName,
                    int versionNumber,
                    boolean alsoWriteStartOfPush)

      • whetherEnableBatchPushFromAdmin

        boolean whetherEnableBatchPushFromAdmin​(java.lang.String storeName)

      • updateAclForStore

        void updateAclForStore​(java.lang.String clusterName,
                       java.lang.String storeName,
                       java.lang.String accessPermisions)
        Provision a new set of ACL for a venice store and its associated kafka topic.

      • getAclForStore

        java.lang.String getAclForStore​(java.lang.String clusterName,
                                java.lang.String storeName)
        Fetch the current set of ACL provisioned for a venice store and its associated kafka topic.
        Returns:
        The string representation of the accessPermissions. It will return empty string in case store is not present.

      • deleteAclForStore

        void deleteAclForStore​(java.lang.String clusterName,
                       java.lang.String storeName)
        Delete the current set of ACL provisioned for a venice store and its associated kafka topic.

      • isParent

        boolean isParent()
        Check whether the controller works as a parent controller
        Returns:
        true if it works as a parent controller. Otherwise, return false.

      • getChildDataCenterControllerUrlMap

        java.util.Map<java.lang.String,​java.lang.String> getChildDataCenterControllerUrlMap​(java.lang.String clusterName)
        Get child datacenter to child controller url mapping.
        Returns:
        A map of child datacenter -> child controller url

      • getChildDataCenterControllerD2Map

        java.util.Map<java.lang.String,​java.lang.String> getChildDataCenterControllerD2Map​(java.lang.String clusterName)
        Get child datacenter to child controller d2 zk host mapping
        Returns:
        A map of child datacenter -> child controller d2 zk host

      • getChildControllerD2ServiceName

        java.lang.String getChildControllerD2ServiceName​(java.lang.String clusterName)
        Get child datacenter controller d2 service name
        Returns:
        d2 service name

      • getClustersLeaderOf

        java.util.List<java.lang.String> getClustersLeaderOf()
        Get a list of clusters this controller is a leader of.
        Returns:
        a list of clusters this controller is a leader of.

      • configureActiveActiveReplication

        void configureActiveActiveReplication​(java.lang.String cluster,
                                      VeniceUserStoreType storeType,
                                      java.util.Optional<java.lang.String> storeName,
                                      boolean enableActiveActiveReplicationForCluster,
                                      java.util.Optional<java.lang.String> regionsFilter)
        Enable/disable active active replications for certain stores (batch only, hybrid only, incremental push, hybrid or incremental push, all) in a cluster. If storeName is not empty, only the specified store might be updated.

      • checkResourceCleanupBeforeStoreCreation

        void checkResourceCleanupBeforeStoreCreation​(java.lang.String clusterName,
                                             java.lang.String storeName)
        Check whether there are any resource left for the store creation in cluster: {@param clusterName} If there is any, this function should throw Exception.

      • getEmergencySourceRegion

        java.util.Optional<java.lang.String> getEmergencySourceRegion​(java.lang.String clusterName)
        Return the emergency source region configuration.

      • getAggregateRealTimeTopicSource

        java.util.Optional<java.lang.String> getAggregateRealTimeTopicSource​(java.lang.String clusterName)
        Return the source Kafka boostrap server url for aggregate real-time topic updates

      • isActiveActiveReplicationEnabledInAllRegion

        boolean isActiveActiveReplicationEnabledInAllRegion​(java.lang.String clusterName,
                                                    java.lang.String storeName,
                                                    boolean checkCurrentVersion)
        Returns true if A/A replication is enabled in all child controller and parent controller. This is implemented only in parent controller. Otherwise, return false.

      • getBackupVersionDefaultRetentionMs

        long getBackupVersionDefaultRetentionMs()
        Returns default backup version retention time.

      • getDefaultMaxRecordSizeBytes

        int getDefaultMaxRecordSizeBytes()
        Returns:
        The default value of VeniceWriter.maxRecordSizeBytes which is provided to the VPJ and Consumer as a controller config to dynamically control the setting per cluster.

      • wipeCluster

        void wipeCluster​(java.lang.String clusterName,
                 java.lang.String fabric,
                 java.util.Optional<java.lang.String> storeName,
                 java.util.Optional<java.lang.Integer> versionNum)

      • copyOverStoreSchemasAndConfigs

        StoreInfo copyOverStoreSchemasAndConfigs​(java.lang.String clusterName,
                                         java.lang.String srcFabric,
                                         java.lang.String destFabric,
                                         java.lang.String storeName)

      • compareStore

        StoreComparisonInfo compareStore​(java.lang.String clusterName,
                                 java.lang.String storeName,
                                 java.lang.String fabricA,
                                 java.lang.String fabricB)
                          throws java.io.IOException
        Compare store metadata and version states between two fabrics.
        Throws:
        java.io.IOException

      • nodeReplicaReadiness

        Pair<NodeReplicasReadinessState,​java.util.List<Replica>> nodeReplicaReadiness​(java.lang.String cluster,
                                                                                    java.lang.String helixNodeId)
        helixNodeId nodeId of helix participant. HOST_PORT. Returns ture, if all current version replicas of the input node are ready to serve. false and all unready replicas otherwise.

      • initiateDataRecovery

        void initiateDataRecovery​(java.lang.String clusterName,
                          java.lang.String storeName,
                          int version,
                          java.lang.String sourceFabric,
                          java.lang.String destinationFabric,
                          boolean copyAllVersionConfigs,
                          java.util.Optional<Version> sourceFabricVersion)
        Initiate data recovery for a store version given a source fabric.
        Parameters:
        clusterName - of the store.
        storeName - of the store.
        version - of the store.
        sourceFabric - to be used as the source for data recovery.
        copyAllVersionConfigs - a boolean to indicate whether all version configs should be copied from the source fabric or only the essential version configs and generate the rest based on destination fabric's Store configs.
        sourceFabricVersion - source fabric's Version configs used to configure the recovering version in the destination fabric.

      • prepareDataRecovery

        void prepareDataRecovery​(java.lang.String clusterName,
                         java.lang.String storeName,
                         int version,
                         java.lang.String sourceFabric,
                         java.lang.String destinationFabric,
                         java.util.Optional<java.lang.Integer> sourceAmplificationFactor)
        Prepare for data recovery in the destination fabric. The interested store version might have lingering states and resources in the destination fabric from previous failed attempts. Perform some basic checks to make sure the store version in the destination fabric is capable of performing data recovery and cleanup any lingering states and resources.

      • isStoreVersionReadyForDataRecovery

        Pair<java.lang.Boolean,​java.lang.String> isStoreVersionReadyForDataRecovery​(java.lang.String clusterName,
                                                                                  java.lang.String storeName,
                                                                                  int version,
                                                                                  java.lang.String sourceFabric,
                                                                                  java.lang.String destinationFabric,
                                                                                  java.util.Optional<java.lang.Integer> sourceAmplificationFactor)
        Check if the store version's previous states and resources are cleaned up and ready to start data recovery.
        Returns:
        whether is ready to start data recovery and the reason if it's not ready.

      • isAdminTopicConsumptionEnabled

        default boolean isAdminTopicConsumptionEnabled​(java.lang.String clusterName)
        Return whether the admin consumption task is enabled for the passed cluster.

      • getClusterStores

        java.util.ArrayList<StoreInfo> getClusterStores​(java.lang.String clusterName)
        Return all stores in a cluster.

      • getClusterStaleStores

        java.util.Map<java.lang.String,​StoreDataAudit> getClusterStaleStores​(java.lang.String clusterName)

      • getLargestUsedVersionFromStoreGraveyard

        int getLargestUsedVersionFromStoreGraveyard​(java.lang.String clusterName,
                                            java.lang.String storeName)
        Returns:
        the largest used version number for the given store from store graveyard.

      • listStorePushInfo

        java.util.Map<java.lang.String,​RegionPushDetails> listStorePushInfo​(java.lang.String clusterName,
                                                                          java.lang.String storeName,
                                                                          boolean isPartitionDetailEnabled)

      • getRegionPushDetails

        RegionPushDetails getRegionPushDetails​(java.lang.String clusterName,
                                       java.lang.String storeName,
                                       boolean isPartitionDetailEnabled)

      • getAdminTopicMetadata

        java.util.Map<java.lang.String,​java.lang.Long> getAdminTopicMetadata​(java.lang.String clusterName,
                                                                           java.util.Optional<java.lang.String> storeName)

      • updateAdminTopicMetadata

        void updateAdminTopicMetadata​(java.lang.String clusterName,
                              long executionId,
                              java.util.Optional<java.lang.String> storeName,
                              java.util.Optional<java.lang.Long> offset,
                              java.util.Optional<java.lang.Long> upstreamOffset)

      • createStoragePersona

        void createStoragePersona​(java.lang.String clusterName,
                          java.lang.String name,
                          long quotaNumber,
                          java.util.Set<java.lang.String> storesToEnforce,
                          java.util.Set<java.lang.String> owners)

      • getStoragePersona

        StoragePersona getStoragePersona​(java.lang.String clusterName,
                                 java.lang.String name)

      • deleteStoragePersona

        void deleteStoragePersona​(java.lang.String clusterName,
                          java.lang.String name)

      • getPersonaAssociatedWithStore

        StoragePersona getPersonaAssociatedWithStore​(java.lang.String clusterName,
                                             java.lang.String storeName)

      • getClusterStoragePersonas

        java.util.List<StoragePersona> getClusterStoragePersonas​(java.lang.String clusterName)

      • cleanupInstanceCustomizedStates

        java.util.List<java.lang.String> cleanupInstanceCustomizedStates​(java.lang.String clusterName)
        Scan through instance level customized states and remove any lingering ZNodes that are no longer relevant. This operation shouldn't be needed under normal circumstances. It's intended to cleanup ZNodes that failed to be deleted due to bugs and errors.
        Parameters:
        clusterName - to perform the cleanup.
        Returns:
        list of deleted ZNode paths.

      • removeStoreFromGraveyard

        void removeStoreFromGraveyard​(java.lang.String clusterName,
                              java.lang.String storeName)

      • startInstanceMonitor

        default void startInstanceMonitor​(java.lang.String clusterName)

      • clearInstanceMonitor

        default void clearInstanceMonitor​(java.lang.String clusterName)

      • sendHeartbeatToSystemStore

        void sendHeartbeatToSystemStore​(java.lang.String clusterName,
                                java.lang.String storeName,
                                long heartbeatTimestamp)
        Send a heartbeat timestamp to targeted system store.

      • getHeartbeatFromSystemStore

        long getHeartbeatFromSystemStore​(java.lang.String clusterName,
                                 java.lang.String storeName)
        Read the latest heartbeat timestamp from system store. If it failed to read from system store, this method should return -1.

      • getHelixVeniceClusterResources

        HelixVeniceClusterResources getHelixVeniceClusterResources​(java.lang.String cluster)
        Returns:
        the aggregate resources required by controller to manage a Venice cluster.