Package com.linkedin.venice.utils.consistency

Class DiffValidationUtils

java.lang.Object
com.linkedin.venice.utils.consistency.DiffValidationUtils
public final class DiffValidationUtils extends Object

  • Method Details

    • isRecordMissing

      public static boolean isRecordMissing(List<Long> existingValueOffsetVector, List<Long> nonExistentValuePartitionOffsetWatermark)
      This method determines if a given record in a snapshot has been lost. It determines this by looking at the records ReplicationMetadata and comparing it to the highwatermark of the colo which did not have this record. If the colo which does not have this record has advanced beyond the offset vector held by the individual record, then we know that this record has been passed by. This function should not be used on any record which holds a tombstone (as the colo which is farther ahead will have purged out a record which currently holds a tombstone).
      Parameters:
      existingValueOffsetVector -
      nonExistentValuePartitionOffsetWatermark -
      Returns:
      True if the record seems to be missing.

    • doRecordsDiverge

      public static boolean doRecordsDiverge(String firstValueChecksum, String secondValueChecksum, List<Long> firstValuePartitionOffsetWatermark, List<Long> secondValuePartitionOffsetWatermark, List<Long> firstValueOffsetVector, List<Long> secondValueOffsetVector)
      This method determines that if given the values for two equal keys in a snapshot of venice data, have the records diverged. It does this, by determining if each colo has received all data which is pertinent to coming to the current resolution held by each snapshot. In other words, does each colo have all the same information to come to the same conclusion, and do these conclusions diverge? This method DOES NOT catch divergence for 'missing' records. Missing records should be determined with {

    • hasOffsetAdvanced

      public static boolean hasOffsetAdvanced(List<Long> baseOffset, List<Long> advancedOffset)
      Checks to see if an offset vector has advanced completely beyond some base offset vector or not.
      Parameters:
      baseOffset - The vector to compare against.
      advancedOffset - The vector has should be advanced along.
      Returns:
      True if the advancedOffset vector has grown beyond the baseOffset