Package com.linkedin.venice.hadoop.input.kafka

Class KafkaInputDictTrainer

java.lang.Object

com.linkedin.venice.hadoop.input.kafka.KafkaInputDictTrainer

public class KafkaInputDictTrainer
extends java.lang.Object

Zstd dict trainer for Kafka Repush. This class will try to read a few records from every partition as samples of dict trainer.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	KafkaInputDictTrainer.Param
static class	KafkaInputDictTrainer.ParamBuilder

Constructor Summary

Constructors

Modifier	Constructor	Description
	KafkaInputDictTrainer(KafkaInputDictTrainer.Param param)
protected	KafkaInputDictTrainer(KafkaInputFormat inputFormat, java.util.Optional<com.github.luben.zstd.ZstdDictTrainer> trainerSupplier, KafkaInputDictTrainer.Param param, com.linkedin.venice.hadoop.input.kafka.KafkaInputDictTrainer.CompressorBuilder compressorBuilder)

Method Summary

Modifier and Type	Method	Description
byte[]	trainDict()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KafkaInputDictTrainer

public KafkaInputDictTrainer(KafkaInputDictTrainer.Param param)

KafkaInputDictTrainer

protected KafkaInputDictTrainer(KafkaInputFormat inputFormat,
                                java.util.Optional<com.github.luben.zstd.ZstdDictTrainer> trainerSupplier,
                                KafkaInputDictTrainer.Param param,
                                com.linkedin.venice.hadoop.input.kafka.KafkaInputDictTrainer.CompressorBuilder compressorBuilder)

Method Detail

trainDict

public byte[] trainDict()