Class AbstractPartitionWriter.DuplicateKeyPrinter

  • All Implemented Interfaces:, java.lang.AutoCloseable
    Enclosing class:

    public static class AbstractPartitionWriter.DuplicateKeyPrinter
    extends java.lang.Object
    implements java.lang.AutoCloseable,
    Using Avro Json encoder to print duplicate keys in case there are tons of duplicate keys, only print first MAX_NUM_OF_LOG of them so that it won't pollute Reducer's log. N.B. We assume that this is an Avro record here. (Vson is considered as Avro as well from Reducer's perspective) We should update this method once Venice supports other format in the future
    • Method Detail

      • detectAndHandleDuplicateKeys

        protected void detectAndHandleDuplicateKeys​(byte[] keyBytes,
                                                    byte[] valueBytes,
                                                    java.util.Iterator<byte[]> values,
                                                    DataWriterTaskTracker dataWriterTaskTracker)
      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface