AbstractMapReduceTask |
|
AbstractVeniceFilter<INPUT_VALUE> |
An abstraction to filter given data type.
|
AbstractVeniceMapper<INPUT_KEY,INPUT_VALUE> |
An abstraction of the mapper that would return serialized, and potentially
compressed, Avro key/value pairs.
|
AbstractVeniceRecordReader<INPUT_KEY,INPUT_VALUE> |
|
DefaultHadoopJobClientProvider |
|
DefaultInputDataInfoProvider |
|
DefaultJobClientWrapper |
|
FilterChain<INPUT_VALUE> |
The FilterChain class takes a list of AbstractVeniceFilter to assemble a filter chain to manage the life cycles'
of filters and perform filtering based on the order of filters.
|
InputDataInfoProvider.InputDataInfo |
|
MRJobCounterHelper |
All MapReducer job counters (identified by a group name + a counter name) are defined here and this util class
provides method to increment counters and read counters
|
NoOpSentPushJobDetailsTracker |
|
PushJobSchemaInfo |
|
PushJobZstdConfig |
|
ValidateSchemaAndBuildDictMapper |
Mapper only MR to Validate Schema, Build compression dictionary if needed and persist
some data (total file size and compression dictionary) in HDFS to be used by the VPJ Driver
Note: processing all the files in this split are done sequentially and if it
results in significant increase in the mapper time or resulting in timeouts,
this needs to be revisited to be done via a thread pool.
|
ValidateSchemaAndBuildDictMapperOutputReader |
|
ValidateSchemaAndBuildDictOutputFormat |
This class provides a way to:
1.
|
VeniceAvroMapper |
|
VeniceAvroRecordReader |
|
VeniceFileInputFormat |
|
VeniceMRPartitioner |
|
VenicePushJob |
This class sets up the Hadoop job used to push data to Venice.
|
VenicePushJob.PushJobSetting |
|
VenicePushJob.StoreSetting |
|
VenicePushJob.TopicInfo |
|
VeniceReducer |
VeniceReducer will be in charge of producing the messages to Kafka broker.
|
VeniceReducer.DuplicateKeyPrinter |
|
VeniceReducer.VeniceWriterMessage |
|
VeniceVsonMapper |
Mapper that reads Vson input and deserializes it as Avro object and then Avro binary
|
VeniceVsonRecordReader |
|
VsonSequenceFileInputFormat |
This class was originally from Voldemort.
|