Class LeaderFollowerStoreIngestionTask

  • All Implemented Interfaces:, java.lang.AutoCloseable, java.lang.Runnable
    Direct Known Subclasses:

    public class LeaderFollowerStoreIngestionTask
    extends StoreIngestionTask
    This class contains the state transition work between leader and follower; both leader and follower will keep track of information like which topic leader is consuming from and the corresponding offset as well as the latest successfully consumed or produced offset in the version topic (VT). State Transition: 1. OFFLINE -> STANDBY: Generate a SUBSCRIBE message in the consumer action queue; the logic here is the same as Online/Offline model; all it needs to do is to restore the checkpointed state from OffsetRecord; 2. STANDBY -> LEADER: The partition will be marked as in the transition progress from STANDBY to LEADER and completes the action immediately; after processing the rest of the consumer actions in the queue, check whether there is any partition is in the transition progress, if so: (i) consume the latest messages from version topic; (ii) drain all the messages in drainer queue in order to update the latest consumed message replication metadata; (iii) check whether there has been at least 5 minutes (configurable) of inactivity for this partition (meaning no new messages); if so, turn on the LEADER flag for this partition. 3. LEADER -> STANDBY: a. if the leader is consuming from VT, just set "isLeader" field to false and resume consumption; b. if the leader is consuming from anything other than VT, it needs to unsubscribe from the leader topic for this partition first, drain all the messages in the drainer queue for this leader topic/partition so that it can get the last producer callback for the last message it produces to VT; block on getting the result from the callback to update the corresponding offset in version topic, so that the new follower can subscribe back to VT using the recently updated VT offset.