Package com.linkedin.venice.helix

Class VeniceOfflinePushMonitorAccessor

  • All Implemented Interfaces:
    OfflinePushAccessor
    public class VeniceOfflinePushMonitorAccessor
extends java.lang.Object
implements OfflinePushAccessor
    Venice implementation of OfflinePushAccessor. All the statuses would be stored on Zookeeper and this class provides the ways to read/write/create/remove status from ZK.

    As this class is only an accessor but not a repository so it will not cache anything in local memory. In other words it's stateless and Thread-Safe.

    The data structure on ZK would be:

    • /OfflinePushes/$topic -> push status for $topic
    • /OfflinePushes/$topic/$partitionId -> partition status including all of replicas statuses for $topic and $partitionId.

    • Field Detail

      • OFFLINE_PUSH_SUB_PATH

        public static final java.lang.String OFFLINE_PUSH_SUB_PATH
        See Also:
        Constant Field Values

    • Constructor Detail

      • VeniceOfflinePushMonitorAccessor

        public VeniceOfflinePushMonitorAccessor​(java.lang.String clusterName,
                                        org.apache.helix.zookeeper.impl.client.ZkClient zkClient,
                                        HelixAdapterSerializer adapter)

      • VeniceOfflinePushMonitorAccessor

        public VeniceOfflinePushMonitorAccessor​(java.lang.String clusterName,
                                        org.apache.helix.zookeeper.impl.client.ZkClient zkClient,
                                        HelixAdapterSerializer adapter,
                                        int refreshAttemptsForZkReconnect,
                                        long refreshIntervalForZkReconnectInMs)

      • VeniceOfflinePushMonitorAccessor

        public VeniceOfflinePushMonitorAccessor​(java.lang.String clusterName,
                                        org.apache.helix.manager.zk.ZkBaseDataAccessor<OfflinePushStatus> offlinePushStatusAccessor,
                                        org.apache.helix.manager.zk.ZkBaseDataAccessor<PartitionStatus> partitionStatusAccessor)
        For testing purpose only.

    • Method Detail

      • loadOfflinePushStatusPaths

        public java.util.List<java.lang.String> loadOfflinePushStatusPaths()
        Description copied from interface: OfflinePushAccessor
        Load store versions of all existing push statuses on Zookeeper, without actually deserializing the contents
        Specified by:
        loadOfflinePushStatusPaths in interface OfflinePushAccessor

      • getOfflinePushStatusCreationTime

        public java.util.Optional<java.lang.Long> getOfflinePushStatusCreationTime​(java.lang.String kafkaTopic)
        Description copied from interface: OfflinePushAccessor
        Get the creation time of the given offline push ZK path; the creation time is set by System.currentTimeMillis(). If the path does not exist or error when trying to get creation time, return an empty Optional.
        Specified by:
        getOfflinePushStatusCreationTime in interface OfflinePushAccessor

      • updateReplicaStatus

        public void updateReplicaStatus​(java.lang.String topic,
                                int partitionId,
                                java.lang.String instanceId,
                                ExecutionStatus status,
                                long progress,
                                java.lang.String incrementalPushVersion)
        Description copied from interface: OfflinePushAccessor
        Update one particular replica status and progress by given topic, partition and instanceId to the persistent storage.
        Specified by:
        updateReplicaStatus in interface OfflinePushAccessor

      • updateReplicaStatus

        public void updateReplicaStatus​(java.lang.String topic,
                                int partitionId,
                                java.lang.String instanceId,
                                ExecutionStatus status,
                                java.lang.String incrementalPushVersion)
        Description copied from interface: OfflinePushAccessor
        Update one particular replica status only by given topic, partition and instanceId to the persistent storage.
        Specified by:
        updateReplicaStatus in interface OfflinePushAccessor

      • batchUpdateReplicaIncPushStatus

        public void batchUpdateReplicaIncPushStatus​(java.lang.String kafkaTopic,
                                            int partitionId,
                                            java.lang.String instanceId,
                                            long progress,
                                            java.util.List<java.lang.String> pendingReportIncPushVersionList)
        Specified by:
        batchUpdateReplicaIncPushStatus in interface OfflinePushAccessor

      • subscribePushStatusCreationChange

        public void subscribePushStatusCreationChange​(org.apache.helix.zookeeper.zkclient.IZkChildListener childListener)
        Description copied from interface: OfflinePushAccessor
        Subscribe a child listener that listens to OfflinePushStatus creation/deleted.
        Specified by:
        subscribePushStatusCreationChange in interface OfflinePushAccessor

      • getPartitionStatus

        protected PartitionStatus getPartitionStatus​(java.lang.String topic,
                                             int partitionId)
        Get one partition status ZNode from ZK by given topic and partition.

      • getPartitionStatuses

        protected java.util.List<PartitionStatus> getPartitionStatuses​(java.lang.String topic,
                                                               int partitionCount)
        Get all partition status ZNodes under offline push of given topic from ZK. The partition statuses paths for a topic are created in ZK one by one; Helix doesn't guarantee that all the paths are created atomically; therefore, it's possible that a partial list is returned. This function would take cover of all edges cases -- empty response as well as partial response by filling the missing partitions. The returned partition status list is ordered by partition Id.

      • getOfflinePushStatuesParentPath

        public final java.lang.String getOfflinePushStatuesParentPath()

      • getOfflinePushStatusAccessor

        public org.apache.helix.manager.zk.ZkBaseDataAccessor<OfflinePushStatus> getOfflinePushStatusAccessor()
        For testing only.