Package com.linkedin.venice.utils.consistency

Class DiffValidationUtils

  • public final class DiffValidationUtils
extends java.lang.Object

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static boolean doRecordsDiverge​(java.lang.String firstValueChecksum, java.lang.String secondValueChecksum, java.util.List<java.lang.Long> firstValuePartitionOffsetWatermark, java.util.List<java.lang.Long> secondValuePartitionOffsetWatermark, java.util.List<java.lang.Long> firstValueOffsetVector, java.util.List<java.lang.Long> secondValueOffsetVector)
      This method determines that if given the values for two equal keys in a snapshot of venice data, have the records diverged.
      static boolean hasOffsetAdvanced​(java.util.List<java.lang.Long> baseOffset, java.util.List<java.lang.Long> advancedOffset)
      Checks to see if an offset vector has advanced completely beyond some base offset vector or not.
      static boolean isRecordMissing​(java.util.List<java.lang.Long> existingValueOffsetVector, java.util.List<java.lang.Long> nonExistentValuePartitionOffsetWatermark)
      This method determines if a given record in a snapshot has been lost.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • isRecordMissing

        public static boolean isRecordMissing​(java.util.List<java.lang.Long> existingValueOffsetVector,
                                      java.util.List<java.lang.Long> nonExistentValuePartitionOffsetWatermark)
        This method determines if a given record in a snapshot has been lost. It determines this by looking at the records ReplicationMetadata and comparing it to the highwatermark of the colo which did not have this record. If the colo which does not have this record has advanced beyond the offset vector held by the individual record, then we know that this record has been passed by. This function should not be used on any record which holds a tombstone (as the colo which is farther ahead will have purged out a record which currently holds a tombstone).
        Parameters:
        existingValueOffsetVector -
        nonExistentValuePartitionOffsetWatermark -
        Returns:
        True if the record seems to be missing.

      • doRecordsDiverge

        public static boolean doRecordsDiverge​(java.lang.String firstValueChecksum,
                                       java.lang.String secondValueChecksum,
                                       java.util.List<java.lang.Long> firstValuePartitionOffsetWatermark,
                                       java.util.List<java.lang.Long> secondValuePartitionOffsetWatermark,
                                       java.util.List<java.lang.Long> firstValueOffsetVector,
                                       java.util.List<java.lang.Long> secondValueOffsetVector)
        This method determines that if given the values for two equal keys in a snapshot of venice data, have the records diverged. It does this, by determining if each colo has received all data which is pertinent to coming to the current resolution held by each snapshot. In other words, does each colo have all the same information to come to the same conclusion, and do these conclusions diverge? This method DOES NOT catch divergence for 'missing' records. Missing records should be determined with {

      • hasOffsetAdvanced

        public static boolean hasOffsetAdvanced​(java.util.List<java.lang.Long> baseOffset,
                                        java.util.List<java.lang.Long> advancedOffset)
        Checks to see if an offset vector has advanced completely beyond some base offset vector or not.
        Parameters:
        baseOffset - The vector to compare against.
        advancedOffset - The vector has should be advanced along.
        Returns:
        True if the advancedOffset vector has grown beyond the baseOffset