Package com.linkedin.venice.controller

Class VeniceParentHelixAdmin

  • All Implemented Interfaces:
    Admin, java.io.Closeable, java.lang.AutoCloseable
    public class VeniceParentHelixAdmin
extends java.lang.Object
implements Admin
    This class is a wrapper of VeniceHelixAdmin, which will be used in parent controller. There should be only one single Parent Controller, which is the endpoint for all the admin data update. For every admin update operation, it will first push admin operation messages to Kafka, then wait for the admin consumer to consume the message. All validations on the updates should be done before the admin operation message is published to Kafka.

    • Method Detail

      • initStorageCluster

        public void initStorageCluster​(java.lang.String clusterName)
        Initialize Venice storage cluster in Helix by:
        • creating and configuring required properties in Helix.
        • waiting resource's (partial) partition to appear in the external view.
        • making sure admin Kafka topics is created.
        • creating a Venice writer for the cluster.
        Specified by:
        initStorageCluster in interface Admin
        Parameters:
        clusterName - Venice cluster name.

      • isClusterValid

        public boolean isClusterValid​(java.lang.String clusterName)
        Test if a cluster is valid (in Helix cluster list).
        Specified by:
        isClusterValid in interface Admin
        Parameters:
        clusterName - Venice cluster name.
        Returns:
        true if input cluster is in Helix cluster list; false otherwise.

      • deleteValueSchemas

        public void deleteValueSchemas​(java.lang.String clusterName,
                               java.lang.String storeName,
                               java.util.Set<java.lang.Integer> unusedValueSchemaIds)
        Description copied from interface: Admin
        Deletes a store's values schema with ids `except` the ids passed in the argument inuseValueSchemaIds
        Specified by:
        deleteValueSchemas in interface Admin

      • getInUseValueSchemaIds

        public java.util.Set<java.lang.Integer> getInUseValueSchemaIds​(java.lang.String clusterName,
                                                               java.lang.String storeName)
        Specified by:
        getInUseValueSchemaIds in interface Admin

      • createStore

        public void createStore​(java.lang.String clusterName,
                        java.lang.String storeName,
                        java.lang.String owner,
                        java.lang.String keySchema,
                        java.lang.String valueSchema,
                        boolean isSystemStore,
                        java.util.Optional<java.lang.String> accessPermissions)
        Create a store by sending STORE_CREATION admin message to the Kafka admin topic, sending META_SYSTEM_STORE_AUTO_CREATION_VALIDATION admin message, and performing initialization steps for using authorize server to manage ACLs for the input store.
        Specified by:
        createStore in interface Admin

      • deleteStore

        public void deleteStore​(java.lang.String clusterName,
                        java.lang.String storeName,
                        int largestUsedVersionNumber,
                        boolean waitOnRTTopicDeletion)
        Delete a store by sending DELETE_STORE admin message to the Kafka admin topic and clearing all ACLs and release resource for the target store from authorize service.
        Specified by:
        deleteStore in interface Admin

      • hasWritePermissionToBatchJobHeartbeatStore

        public boolean hasWritePermissionToBatchJobHeartbeatStore​(java.security.cert.X509Certificate requesterCert,
                                                          java.lang.String batchJobHeartbeatStoreName)
                                                   throws AclException
        Test if the given certificate has the write-access permission for the given batch-job heartbeat store.
        Specified by:
        hasWritePermissionToBatchJobHeartbeatStore in interface Admin
        Parameters:
        requesterCert - X.509 certificate object.
        batchJobHeartbeatStoreName - name of the batch-job heartbeat store.
        Returns:
        true if input certificate has write-access permission for the given store; false otherwise.
        Throws:
        AclException

      • incrementVersionIdempotent

        public Version incrementVersionIdempotent​(java.lang.String clusterName,
                                          java.lang.String storeName,
                                          java.lang.String pushJobId,
                                          int numberOfPartitions,
                                          int replicationFactor,
                                          Version.PushType pushType,
                                          boolean sendStartOfPush,
                                          boolean sorted,
                                          java.lang.String compressionDictionary,
                                          java.util.Optional<java.lang.String> sourceGridFabric,
                                          java.util.Optional<java.security.cert.X509Certificate> requesterCert,
                                          long rewindTimeInSecondsOverride,
                                          java.util.Optional<java.lang.String> emergencySourceRegion,
                                          boolean versionSwapDeferred,
                                          java.lang.String targetedRegions,
                                          int repushSourceVersion)
        Specified by:
        incrementVersionIdempotent in interface Admin
        See Also:
        Admin.incrementVersionIdempotent(String, String, String, int, int)

      • getIncrementalPushVersion

        public Version getIncrementalPushVersion​(java.lang.String clusterName,
                                         java.lang.String storeName)
        A couple of extra checks are needed in parent controller 1. check batch job statuses across child controllers. (We cannot only check the version status in parent controller since they are marked as STARTED) 2. check if the topic is marked to be truncated or not. (This could be removed if we don't preserve incremental push topic in parent Kafka anymore
        Specified by:
        getIncrementalPushVersion in interface Admin

      • getCurrentVersion

        public int getCurrentVersion​(java.lang.String clusterName,
                             java.lang.String storeName)
        Unsupported operation in the parent controller.
        Specified by:
        getCurrentVersion in interface Admin

      • getCurrentVersionsForMultiColos

        public java.util.Map<java.lang.String,​java.lang.Integer> getCurrentVersionsForMultiColos​(java.lang.String clusterName,
                                                                                               java.lang.String storeName)
        Query the current version for the given store. In parent colo, Venice do not update the current version because there is not offline push monitor. So parent controller will query each prod controller and return the map.
        Specified by:
        getCurrentVersionsForMultiColos in interface Admin

      • getRepushInfo

        public RepushInfo getRepushInfo​(java.lang.String clusterName,
                                java.lang.String storeName,
                                java.util.Optional<java.lang.String> fabricName)
        Specified by:
        getRepushInfo in interface Admin
        Returns:
        a RepushInfo object with store information retrieved from the specified cluster and fabric.

      • getBackupVersionsForMultiColos

        public java.util.Map<java.lang.String,​java.lang.String> getBackupVersionsForMultiColos​(java.lang.String clusterName,
                                                                                             java.lang.String storeName)
        Specified by:
        getBackupVersionsForMultiColos in interface Admin

      • getFutureVersion

        public int getFutureVersion​(java.lang.String clusterName,
                            java.lang.String storeName)
        Unsupported operation in the parent controller and returns Store.NON_EXISTING_VERSION.
        Specified by:
        getFutureVersion in interface Admin

      • getBackupVersion

        public int getBackupVersion​(java.lang.String clusterName,
                            java.lang.String storeName)
        Specified by:
        getBackupVersion in interface Admin

      • peekNextVersion

        public Version peekNextVersion​(java.lang.String clusterName,
                               java.lang.String storeName)
        Unsupported operation in the parent controller.
        Specified by:
        peekNextVersion in interface Admin

      • deleteOldVersionInStore

        public void deleteOldVersionInStore​(java.lang.String clusterName,
                                    java.lang.String storeName,
                                    int versionNum)
        Description copied from interface: Admin
        Delete the given version from the store. If the given version is the current version, an exception will be thrown.
        Specified by:
        deleteOldVersionInStore in interface Admin
        See Also:
        Admin.deleteOldVersionInStore(String, String, int)

      • versionsForStore

        public java.util.List<Version> versionsForStore​(java.lang.String clusterName,
                                                java.lang.String storeName)
        Specified by:
        versionsForStore in interface Admin
        Returns:
        all versions of the specified store from a cluster.

      • getAllStores

        public java.util.List<Store> getAllStores​(java.lang.String clusterName)
        Specified by:
        getAllStores in interface Admin
        Returns:
        all stores in the specified cluster.

      • getAllStoreStatuses

        public java.util.Map<java.lang.String,​java.lang.String> getAllStoreStatuses​(java.lang.String clusterName)
        Unsupported operation in the parent controller.
        Specified by:
        getAllStoreStatuses in interface Admin
        Returns:
        a map whose key is store name and value is store's status.

      • getStore

        public Store getStore​(java.lang.String clusterName,
                      java.lang.String storeName)
        Specified by:
        getStore in interface Admin
        Returns:
        Store object reference from the input store name.

      • setStoreCurrentVersion

        public void setStoreCurrentVersion​(java.lang.String clusterName,
                                   java.lang.String storeName,
                                   int versionNumber)
        Unsupported operation in the parent controller.
        Specified by:
        setStoreCurrentVersion in interface Admin

      • rollForwardToFutureVersion

        public void rollForwardToFutureVersion​(java.lang.String clusterName,
                                       java.lang.String storeName,
                                       java.lang.String regionFilter)
        Specified by:
        rollForwardToFutureVersion in interface Admin

      • rollbackToBackupVersion

        public void rollbackToBackupVersion​(java.lang.String clusterName,
                                    java.lang.String storeName,
                                    java.lang.String regionFilter)
        Set backup version as current version in all child regions.
        Specified by:
        rollbackToBackupVersion in interface Admin

      • setStoreLargestUsedVersion

        public void setStoreLargestUsedVersion​(java.lang.String clusterName,
                                       java.lang.String storeName,
                                       int versionNumber)
        Unsupported operation in the parent controller.
        Specified by:
        setStoreLargestUsedVersion in interface Admin

      • setStoreOwner

        public void setStoreOwner​(java.lang.String clusterName,
                          java.lang.String storeName,
                          java.lang.String owner)
        Update the owner of a specified store by sending SET_STORE_OWNER admin message to the admin topic.
        Specified by:
        setStoreOwner in interface Admin

      • setStorePartitionCount

        public void setStorePartitionCount​(java.lang.String clusterName,
                                   java.lang.String storeName,
                                   int partitionCount)
        Update the partition count of a specified store by sending SET_STORE_PARTITION admin message to the admin topic.
        Specified by:
        setStorePartitionCount in interface Admin

      • setStoreReadability

        public void setStoreReadability​(java.lang.String clusterName,
                                java.lang.String storeName,
                                boolean desiredReadability)
        Update the readability of a specified store by sending ENABLE_STORE_READ or DISABLE_STORE_READ admin message.
        Specified by:
        setStoreReadability in interface Admin

      • setStoreWriteability

        public void setStoreWriteability​(java.lang.String clusterName,
                                 java.lang.String storeName,
                                 boolean desiredWriteability)
        Update the writability of a specified store by sending ENABLE_STORE_WRITE or DISABLE_STORE_WRITE admin message.
        Specified by:
        setStoreWriteability in interface Admin

      • setStoreReadWriteability

        public void setStoreReadWriteability​(java.lang.String clusterName,
                                     java.lang.String storeName,
                                     boolean isAccessible)
        Update both readability and writability of a specified store.
        Specified by:
        setStoreReadWriteability in interface Admin

      • updateStore

        public void updateStore​(java.lang.String clusterName,
                        java.lang.String storeName,
                        UpdateStoreQueryParams params)
        Update a target store properties by first applying the provided deltas and then sending UPDATE_STORE admin message.
        Specified by:
        updateStore in interface Admin
        Parameters:
        clusterName - name of the Venice cluster.
        storeName - name of the to-be-updated store.
        params - to-be-updated store properties.

      • addValueSchema

        public SchemaEntry addValueSchema​(java.lang.String clusterName,
                                  java.lang.String storeName,
                                  java.lang.String newValueSchemaStr,
                                  DirectionalSchemaCompatibilityType expectedCompatibilityType)
        Add a new value schema for the given store with all specified properties by sending a VALUE_SCHEMA_CREATION admin message.
        Specified by:
        addValueSchema in interface Admin
        Returns:
        an SchemaEntry object composed of a schema and its corresponding id.

      • addSupersetSchema

        public SchemaEntry addSupersetSchema​(java.lang.String clusterName,
                                     java.lang.String storeName,
                                     java.lang.String valueSchemaStr,
                                     int valueSchemaId,
                                     java.lang.String supersetSchemaStr,
                                     int supersetSchemaId)
        Unsupported operation in the parent controller.
        Specified by:
        addSupersetSchema in interface Admin

      • addDerivedSchema

        public DerivedSchemaEntry addDerivedSchema​(java.lang.String clusterName,
                                           java.lang.String storeName,
                                           int valueSchemaId,
                                           java.lang.String derivedSchemaStr)
        Add a new superset schema for the given store with all specified properties by sending a DERIVED_SCHEMA_CREATION admin message.
        Specified by:
        addDerivedSchema in interface Admin
        Returns:
        an DerivedSchemaEntry object composed of a derived schema and its corresponding id.

      • addDerivedSchema

        public DerivedSchemaEntry addDerivedSchema​(java.lang.String clusterName,
                                           java.lang.String storeName,
                                           int valueSchemaId,
                                           int derivedSchemaId,
                                           java.lang.String derivedSchemaStr)
        Unsupported operation in the parent controller.
        Specified by:
        addDerivedSchema in interface Admin

      • removeDerivedSchema

        public DerivedSchemaEntry removeDerivedSchema​(java.lang.String clusterName,
                                              java.lang.String storeName,
                                              int valueSchemaId,
                                              int derivedSchemaId)
        Unsupported operation in the parent controller.
        Specified by:
        removeDerivedSchema in interface Admin
        Returns:
        the derived schema that is deleted or null if the schema doesn't exist

      • addReplicationMetadataSchema

        public RmdSchemaEntry addReplicationMetadataSchema​(java.lang.String clusterName,
                                                   java.lang.String storeName,
                                                   int valueSchemaId,
                                                   int replicationMetadataVersionId,
                                                   java.lang.String replicationMetadataSchemaStr)
        Create a new ReplicationMetadataSchemaEntry object with the given properties and add it into schema repository by sending REPLICATION_METADATA_SCHEMA_CREATION admin message.
        Specified by:
        addReplicationMetadataSchema in interface Admin

      • getStorageNodes

        public java.util.List<java.lang.String> getStorageNodes​(java.lang.String clusterName)
        Unsupported operation in the parent controller.
        Specified by:
        getStorageNodes in interface Admin

      • getStorageNodesStatus

        public java.util.Map<java.lang.String,​java.lang.String> getStorageNodesStatus​(java.lang.String clusterName,
                                                                                    boolean enableReplica)
        Unsupported operation in the parent controller.
        Specified by:
        getStorageNodesStatus in interface Admin

      • removeStorageNode

        public void removeStorageNode​(java.lang.String clusterName,
                              java.lang.String instanceId)
        Unsupported operation in the parent controller.
        Specified by:
        removeStorageNode in interface Admin

      • getOffLinePushStatus

        public Admin.OfflinePushStatusInfo getOffLinePushStatus​(java.lang.String clusterName,
                                                        java.lang.String kafkaTopic)
        Queries child clusters for status. Of all responses, return highest of (in order) NOT_CREATED, NEW, STARTED, PROGRESS. If all responses are COMPLETED, returns COMPLETED. If any response is ERROR and all responses are terminal (COMPLETED or ERROR), returns ERROR If any response is ERROR and any response is not terminal, returns PROGRESS ARCHIVED is treated as NOT_CREATED If error in querying half or more of clusters, returns PROGRESS. (so that polling will continue)
        Specified by:
        getOffLinePushStatus in interface Admin
        Parameters:
        clusterName -
        kafkaTopic -
        Returns:

      • getFinalReturnStatus

        protected static ExecutionStatus getFinalReturnStatus​(java.util.Map<java.lang.String,​ExecutionStatus> statuses,
                                                      java.util.Set<java.lang.String> childRegions,
                                                      int numChildRegionsFailedToFetchStatus,
                                                      java.lang.StringBuilder currentReturnStatusDetails)
        Based on the global information, start determining the final status to return
        Parameters:
        statuses -
        childRegions -
        numChildRegionsFailedToFetchStatus -
        currentReturnStatusDetails -
        Returns:

      • getRegionName

        public java.lang.String getRegionName()
        Description copied from interface: Admin
        Return the region name of this Admin
        Specified by:
        getRegionName in interface Admin
        Returns:
        the region name of this controller

      • isLeaderControllerFor

        public boolean isLeaderControllerFor​(java.lang.String clusterName)
        Description copied from interface: Admin
        Check if this controller itself is the leader controller for a given cluster or not. Note that the controller can be either a parent controller or a child controller since a cluster must have a leader child controller and a leader parent controller. The point is not to be confused the concept of leader-standby with parent-child controller architecture.
        Specified by:
        isLeaderControllerFor in interface Admin
        See Also:
        VeniceHelixAdmin.isLeaderControllerFor(String)

      • getDatacenterCount

        public int getDatacenterCount​(java.lang.String clusterName)
        Description copied from interface: Admin
        number of datacenters, 1 if in single cluster mode. Could be more if this is a parent controller
        Specified by:
        getDatacenterCount in interface Admin
        See Also:
        Admin.getDatacenterCount(String)

      • getReplicasOfStorageNode

        public java.util.List<Replica> getReplicasOfStorageNode​(java.lang.String clusterName,
                                                        java.lang.String instanceId)
        Unsupported operation in the parent controller.
        Specified by:
        getReplicasOfStorageNode in interface Admin

      • isInstanceRemovable

        public NodeRemovableResult isInstanceRemovable​(java.lang.String clusterName,
                                               java.lang.String instanceId,
                                               java.util.List<java.lang.String> lockedNodes,
                                               boolean isFromInstanceView)
        Unsupported operation in the parent controller.
        Specified by:
        isInstanceRemovable in interface Admin
        instanceId - nodeId of helix participant. HOST_PORT.
        lockedNodes - A list of helix nodeIds whose resources are assumed to be unusable (stopped).
        isFromInstanceView - If the value is true, it means we will only check the partitions this instance hold. E.g. if all replicas of a partition are error, but this instance does not hold any replica in this partition, we will skip this partition in the checking. If the value is false, we will check all partitions of resources this instance hold.

      • initiateDataRecovery

        public void initiateDataRecovery​(java.lang.String clusterName,
                                 java.lang.String storeName,
                                 int version,
                                 java.lang.String sourceFabric,
                                 java.lang.String destinationFabric,
                                 boolean copyAllVersionConfigs,
                                 java.util.Optional<Version> ignored)
        Description copied from interface: Admin
        Initiate data recovery for a store version given a source fabric.
        Specified by:
        initiateDataRecovery in interface Admin
        Parameters:
        clusterName - of the store.
        storeName - of the store.
        version - of the store.
        sourceFabric - to be used as the source for data recovery.
        copyAllVersionConfigs - a boolean to indicate whether all version configs should be copied from the source fabric or only the essential version configs and generate the rest based on destination fabric's Store configs.
        ignored - source fabric's Version configs used to configure the recovering version in the destination fabric.
        See Also:
        Admin.initiateDataRecovery(String, String, int, String, String, boolean, Optional)

      • prepareDataRecovery

        public void prepareDataRecovery​(java.lang.String clusterName,
                                java.lang.String storeName,
                                int version,
                                java.lang.String sourceFabric,
                                java.lang.String destinationFabric,
                                java.util.Optional<java.lang.Integer> ignored)
        Description copied from interface: Admin
        Prepare for data recovery in the destination fabric. The interested store version might have lingering states and resources in the destination fabric from previous failed attempts. Perform some basic checks to make sure the store version in the destination fabric is capable of performing data recovery and cleanup any lingering states and resources.
        Specified by:
        prepareDataRecovery in interface Admin
        See Also:
        Admin.prepareDataRecovery(String, String, int, String, String, Optional)

      • isStoreVersionReadyForDataRecovery

        public Pair<java.lang.Boolean,​java.lang.String> isStoreVersionReadyForDataRecovery​(java.lang.String clusterName,
                                                                                         java.lang.String storeName,
                                                                                         int version,
                                                                                         java.lang.String sourceFabric,
                                                                                         java.lang.String destinationFabric,
                                                                                         java.util.Optional<java.lang.Integer> ignored)
        Description copied from interface: Admin
        Check if the store version's previous states and resources are cleaned up and ready to start data recovery.
        Specified by:
        isStoreVersionReadyForDataRecovery in interface Admin
        Returns:
        whether is ready to start data recovery and the reason if it's not ready.
        See Also:
        Admin.isStoreVersionReadyForDataRecovery(String, String, int, String, String, Optional)

      • getLeaderController

        public Instance getLeaderController​(java.lang.String clusterName)
        Description copied from interface: Admin
        Get instance of leader controller. If there is no leader controller for the given cluster, throw a VeniceException.
        Specified by:
        getLeaderController in interface Admin
        See Also:
        Admin.getLeaderController(String)

      • addInstanceToAllowlist

        public void addInstanceToAllowlist​(java.lang.String clusterName,
                                   java.lang.String helixNodeId)
        Unsupported operation in the parent controller.
        Specified by:
        addInstanceToAllowlist in interface Admin

      • removeInstanceFromAllowList

        public void removeInstanceFromAllowList​(java.lang.String clusterName,
                                        java.lang.String helixNodeId)
        Unsupported operation in the parent controller.
        Specified by:
        removeInstanceFromAllowList in interface Admin

      • getAllowlist

        public java.util.Set<java.lang.String> getAllowlist​(java.lang.String clusterName)
        Unsupported operation in the parent controller.
        Specified by:
        getAllowlist in interface Admin

      • killOfflinePush

        public void killOfflinePush​(java.lang.String clusterName,
                            java.lang.String kafkaTopic,
                            boolean isForcedKill)
        Description copied from interface: Admin
        Kill an offline push if it ran into errors or the corresponding version is being retired.
        Specified by:
        killOfflinePush in interface Admin
        isForcedKill - should be set to true when killing the push job for retiring the corresponding version.
        See Also:
        Admin.killOfflinePush(String, String, boolean)

      • getStorageNodesStatus

        public StorageNodeStatus getStorageNodesStatus​(java.lang.String clusterName,
                                               java.lang.String instanceId)
        Unsupported operation in the parent controller.
        Specified by:
        getStorageNodesStatus in interface Admin

      • isStorageNodeNewerOrEqualTo

        public boolean isStorageNodeNewerOrEqualTo​(java.lang.String clusterName,
                                           java.lang.String instanceId,
                                           StorageNodeStatus oldServerStatus)
        Unsupported operation in the parent controller.
        Specified by:
        isStorageNodeNewerOrEqualTo in interface Admin

      • skipAdminMessage

        public void skipAdminMessage​(java.lang.String clusterName,
                             long offset,
                             boolean skipDIV)
        Description copied from interface: Admin
        The admin consumption task tries to deal with failures to process an admin message by retrying. If there is a message that cannot be processed for some reason, we will need to forcibly skip that message in order to unblock the task from consuming subsequent messages.
        Specified by:
        skipAdminMessage in interface Admin
        skipDIV - tries to skip only the DIV check for the blocking message.
        See Also:
        Admin.skipAdminMessage(String, long, boolean)

      • getAdminTopicMetadata

        public java.util.Map<java.lang.String,​java.lang.Long> getAdminTopicMetadata​(java.lang.String clusterName,
                                                                                  java.util.Optional<java.lang.String> storeName)
        Unsupported operation in the parent controller.
        Specified by:
        getAdminTopicMetadata in interface Admin

      • updateAdminTopicMetadata

        public void updateAdminTopicMetadata​(java.lang.String clusterName,
                                     long executionId,
                                     java.util.Optional<java.lang.String> storeName,
                                     java.util.Optional<java.lang.Long> offset,
                                     java.util.Optional<java.lang.Long> upstreamOffset)
        Unsupported operation in the parent controller.
        Specified by:
        updateAdminTopicMetadata in interface Admin

      • updateRoutersClusterConfig

        public void updateRoutersClusterConfig​(java.lang.String clusterName,
                                       java.util.Optional<java.lang.Boolean> isThrottlingEnable,
                                       java.util.Optional<java.lang.Boolean> isQuotaRebalancedEnable,
                                       java.util.Optional<java.lang.Boolean> isMaxCapacityProtectionEnabled,
                                       java.util.Optional<java.lang.Integer> expectedRouterCount)
        Unsupported operation in the parent controller.
        Specified by:
        updateRoutersClusterConfig in interface Admin

      • discoverCluster

        public Pair<java.lang.String,​java.lang.String> discoverCluster​(java.lang.String storeName)
        Description copied from interface: Admin
        Find the cluster which the given store belongs to. Return the pair of the cluster name and the d2 service associated with that cluster.
        Specified by:
        discoverCluster in interface Admin
        See Also:
        Admin.discoverCluster(String)

      • getServerD2Service

        public java.lang.String getServerD2Service​(java.lang.String clusterName)
        Description copied from interface: Admin
        Find the server d2 service associated with a given cluster name.
        Specified by:
        getServerD2Service in interface Admin
        See Also:
        Admin.getServerD2Service(String)

      • findAllBootstrappingVersions

        public java.util.Map<java.lang.String,​java.lang.String> findAllBootstrappingVersions​(java.lang.String clusterName)
        Unsupported operation in the parent controller.
        Specified by:
        findAllBootstrappingVersions in interface Admin

      • getVeniceWriterFactory

        public VeniceWriterFactory getVeniceWriterFactory()
        Specified by:
        getVeniceWriterFactory in interface Admin
        Returns:
        a VeniceWriterFactory object used by the Venice controller to create the venice writer.

      • stop

        public void stop​(java.lang.String clusterName)
        Description copied from interface: Admin
        Stop the helix controller for a single cluster.
        Specified by:
        stop in interface Admin
        See Also:
        Admin.stop(String)

      • close

        public void close()
        Cause VeniceParentHelixAdmin and its associated services to stop executing.
        Specified by:
        close in interface Admin
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable

      • isLeaderControllerOfControllerCluster

        public boolean isLeaderControllerOfControllerCluster()
        Description copied from interface: Admin
        This function can be used to perform cluster-wide operations which need to be performed by a single process only in the whole cluster. There could be a race condition during leader controller failover, and so long operation should have some way of guarding against that.
        Specified by:
        isLeaderControllerOfControllerCluster in interface Admin
        See Also:
        VeniceHelixAdmin.isLeaderControllerOfControllerCluster()

      • isTopicTruncatedBasedOnRetention

        public boolean isTopicTruncatedBasedOnRetention​(java.lang.String kafkaTopicName,
                                                long retentionTime)
        Specified by:
        isTopicTruncatedBasedOnRetention in interface Admin

      • truncateKafkaTopic

        public boolean truncateKafkaTopic​(java.lang.String kafkaTopicName)
        Specified by:
        truncateKafkaTopic in interface Admin
        Returns:
        false indicates that the truncate operation has already been done before; true if it's the first time truncating this topic.
        See Also:
        VeniceHelixAdmin.truncateKafkaTopic(String)

      • truncateKafkaTopic

        public boolean truncateKafkaTopic​(java.lang.String kafkaTopicName,
                                  long retentionTime)
        Description copied from interface: Admin
        Truncate a Kafka topic by setting its retention time to the input value.
        Specified by:
        truncateKafkaTopic in interface Admin
        Parameters:
        kafkaTopicName - the name of the topic to truncate.
        retentionTime - the retention time in milliseconds to set for the topic.
        Returns:
        true if truncating this topic successfully. false otherwise.
        See Also:
        Admin.truncateKafkaTopic(String, long)

      • isResourceStillAlive

        public boolean isResourceStillAlive​(java.lang.String resourceName)
        Unsupported operation in the parent controller.
        Specified by:
        isResourceStillAlive in interface Admin

      • updateClusterDiscovery

        public void updateClusterDiscovery​(java.lang.String storeName,
                                   java.lang.String oldCluster,
                                   java.lang.String newCluster,
                                   java.lang.String initiatingCluster)
        Description copied from interface: Admin
        Update the cluster discovery of a given store by writing to the StoreConfig ZNode.
        Specified by:
        updateClusterDiscovery in interface Admin
        Parameters:
        storeName - of the store.
        oldCluster - for the store.
        newCluster - for the store.
        initiatingCluster - that is making the update. This is needed because in the case of store migration sometimes the update is not made by the leader of the current cluster but instead the leader of the source cluster.
        See Also:
        Admin.updateClusterDiscovery(String, String, String, String)

      • whetherEnableBatchPushFromAdmin

        public boolean whetherEnableBatchPushFromAdmin​(java.lang.String storeName)
        Specified by:
        whetherEnableBatchPushFromAdmin in interface Admin

      • migrateStore

        public void migrateStore​(java.lang.String srcClusterName,
                         java.lang.String destClusterName,
                         java.lang.String storeName)
        Migrate a store from its source cluster to a new destination cluster by sending a MIGRATE_STORE admin message.
        Specified by:
        migrateStore in interface Admin

      • abortMigration

        public void abortMigration​(java.lang.String srcClusterName,
                           java.lang.String destClusterName,
                           java.lang.String storeName)
        Abort store migration by sending a ABORT_MIGRATION admin message.
        Specified by:
        abortMigration in interface Admin

      • updateAclForStore

        public void updateAclForStore​(java.lang.String clusterName,
                              java.lang.String storeName,
                              java.lang.String accessPermissions)
        Description copied from interface: Admin
        Provision a new set of ACL for a venice store and its associated kafka topic.
        Specified by:
        updateAclForStore in interface Admin
        See Also:
        Admin.updateAclForStore(String, String, String)

      • updateSystemStoreAclForStore

        public void updateSystemStoreAclForStore​(java.lang.String clusterName,
                                         java.lang.String regularStoreName,
                                         AclBinding systemStoreAclBinding)
        Set the AceEntries in provided AclBinding object to be the current set of ACL's for the resource.

      • getAclForStore

        public java.lang.String getAclForStore​(java.lang.String clusterName,
                                       java.lang.String storeName)
        Description copied from interface: Admin
        Fetch the current set of ACL provisioned for a venice store and its associated kafka topic.
        Specified by:
        getAclForStore in interface Admin
        Returns:
        The string representation of the accessPermissions. It will return empty string in case store is not present.
        See Also:
        Admin.getAclForStore(String, String)

      • deleteAclForStore

        public void deleteAclForStore​(java.lang.String clusterName,
                              java.lang.String storeName)
        Description copied from interface: Admin
        Delete the current set of ACL provisioned for a venice store and its associated kafka topic.
        Specified by:
        deleteAclForStore in interface Admin
        See Also:
        Admin.deleteAclForStore(String, String)

      • getClusterStaleStores

        public java.util.Map<java.lang.String,​StoreDataAudit> getClusterStaleStores​(java.lang.String clusterName)
        This function will iterate over all of Helix Parent Admin's child controllers, in order to ask about stale stores.
        Specified by:
        getClusterStaleStores in interface Admin

      • getLargestUsedVersionFromStoreGraveyard

        public int getLargestUsedVersionFromStoreGraveyard​(java.lang.String clusterName,
                                                   java.lang.String storeName)
        Specified by:
        getLargestUsedVersionFromStoreGraveyard in interface Admin
        Returns:
        the largest used version number for the given store from the store graveyard.

      • getClusterStores

        public java.util.ArrayList<StoreInfo> getClusterStores​(java.lang.String clusterName)
        Unsupported operation in the parent controller.
        Specified by:
        getClusterStores in interface Admin

      • getRegionPushDetails

        public RegionPushDetails getRegionPushDetails​(java.lang.String clusterName,
                                              java.lang.String storeName,
                                              boolean isPartitionDetailAdded)
        Unsupported operation in the parent controller.
        Specified by:
        getRegionPushDetails in interface Admin

      • listStorePushInfo

        public java.util.Map<java.lang.String,​RegionPushDetails> listStorePushInfo​(java.lang.String clusterName,
                                                                                 java.lang.String storeName,
                                                                                 boolean isPartitionDetailEnabled)
        This function will look for a single store, given a name and cluster name, and return information about the current push jobs for that store across all regions.
        Specified by:
        listStorePushInfo in interface Admin

      • checkResourceCleanupBeforeStoreCreation

        public void checkResourceCleanupBeforeStoreCreation​(java.lang.String clusterName,
                                                    java.lang.String storeName)
        This function will check whether there are still resources left for the requested store in the requested cluster. This function will check both parent colo and all prod colos.
        Specified by:
        checkResourceCleanupBeforeStoreCreation in interface Admin

      • isParent

        public boolean isParent()
        Description copied from interface: Admin
        Check whether the controller works as a parent controller
        Specified by:
        isParent in interface Admin
        Returns:
        true if it works as a parent controller. Otherwise, return false.
        See Also:
        Admin.isParent()

      • getChildDataCenterControllerD2Map

        public java.util.Map<java.lang.String,​java.lang.String> getChildDataCenterControllerD2Map​(java.lang.String clusterName)
        Description copied from interface: Admin
        Get child datacenter to child controller d2 zk host mapping
        Specified by:
        getChildDataCenterControllerD2Map in interface Admin
        Returns:
        A map of child datacenter -> child controller d2 zk host
        See Also:
        Admin.getChildDataCenterControllerD2Map(String)

      • getClustersLeaderOf

        public java.util.List<java.lang.String> getClustersLeaderOf()
        Description copied from interface: Admin
        Get a list of clusters this controller is a leader of.
        Specified by:
        getClustersLeaderOf in interface Admin
        Returns:
        a list of clusters this controller is a leader of.
        See Also:
        Admin.getClustersLeaderOf()

      • getDefaultMaxRecordSizeBytes

        public int getDefaultMaxRecordSizeBytes()
        Specified by:
        getDefaultMaxRecordSizeBytes in interface Admin
        Returns:
        The default value of VeniceWriter.maxRecordSizeBytes which is provided to the VPJ and Consumer as a controller config to dynamically control the setting per cluster.
        See Also:
        Admin.getDefaultMaxRecordSizeBytes()

      • deleteStoragePersona

        public void deleteStoragePersona​(java.lang.String clusterName,
                                 java.lang.String name)
        Deletes the persona with the given name. If no persona is found, this method does nothing.
        Specified by:
        deleteStoragePersona in interface Admin

      • cleanupInstanceCustomizedStates

        public java.util.List<java.lang.String> cleanupInstanceCustomizedStates​(java.lang.String clusterName)
        Description copied from interface: Admin
        Scan through instance level customized states and remove any lingering ZNodes that are no longer relevant. This operation shouldn't be needed under normal circumstances. It's intended to cleanup ZNodes that failed to be deleted due to bugs and errors.
        Specified by:
        cleanupInstanceCustomizedStates in interface Admin
        Parameters:
        clusterName - to perform the cleanup.
        Returns:
        list of deleted ZNode paths.

      • removeStoreFromGraveyard

        public void removeStoreFromGraveyard​(java.lang.String clusterName,
                                     java.lang.String storeName)
        Specified by:
        removeStoreFromGraveyard in interface Admin

      • sendHeartbeatToSystemStore

        public void sendHeartbeatToSystemStore​(java.lang.String clusterName,
                                       java.lang.String systemStoreName,
                                       long heartbeatTimestamp)
        Description copied from interface: Admin
        Send a heartbeat timestamp to targeted system store.
        Specified by:
        sendHeartbeatToSystemStore in interface Admin

      • getHeartbeatFromSystemStore

        public long getHeartbeatFromSystemStore​(java.lang.String clusterName,
                                        java.lang.String storeName)
        Description copied from interface: Admin
        Read the latest heartbeat timestamp from system store. If it failed to read from system store, this method should return -1.
        Specified by:
        getHeartbeatFromSystemStore in interface Admin