Package com.linkedin.venice.spark.datawriter.partition

Class PartitionSorter

java.lang.Object

com.linkedin.venice.spark.datawriter.partition.PartitionSorter

All Implemented Interfaces:

Serializable, Comparator<org.apache.spark.sql.Row>

public class PartitionSorter
extends Object
implements Comparator<org.apache.spark.sql.Row>, Serializable

Sort the rows based on the key and value in ascending order using unsigned byte comparison.

• The sorting on the key is the same as what RocksDB and Shuffle-Sort in MapReduce use.
• The sorting on the value is to make SparkPartitionWriter be able to optimize the de-duping of values.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
PartitionSorter()

Method Summary

Modifier and Type	Method	Description
int	compare(org.apache.spark.sql.Row r1, org.apache.spark.sql.Row r2)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Comparator

equals, reversed, thenComparing, thenComparing, thenComparing, thenComparingDouble, thenComparingInt, thenComparingLong

Constructor Details

PartitionSorter

public PartitionSorter()

Method Details

compare

public int compare(org.apache.spark.sql.Row r1,
 org.apache.spark.sql.Row r2)

Specified by:

compare in interface Comparator<org.apache.spark.sql.Row>