Package com.linkedin.venice.hadoop.mapreduce.datawriter.map

Class AbstractVeniceMapper<INPUT_KEY,INPUT_VALUE>

java.lang.Object

com.linkedin.venice.hadoop.task.datawriter.AbstractDataWriterTask

com.linkedin.venice.hadoop.task.datawriter.AbstractInputRecordProcessor<INPUT_KEY,INPUT_VALUE>

com.linkedin.venice.hadoop.mapreduce.datawriter.map.AbstractVeniceMapper<INPUT_KEY,INPUT_VALUE>

Type Parameters:

INPUT_KEY - type of the input key read from InputFormat

INPUT_VALUE - type of the input value read from InputFormat

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.io.Closeable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Mapper<INPUT_KEY,INPUT_VALUE,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Direct Known Subclasses:

VeniceAvroMapper, VeniceKafkaInputMapper, VeniceVsonMapper

public abstract class AbstractVeniceMapper<INPUT_KEY,INPUT_VALUE>
extends AbstractInputRecordProcessor<INPUT_KEY,INPUT_VALUE>
implements org.apache.hadoop.mapred.Mapper<INPUT_KEY,INPUT_VALUE,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

An abstraction of the mapper that would return serialized, and potentially compressed, Avro key/value pairs.

Field Summary

Fields inherited from class com.linkedin.venice.hadoop.task.datawriter.AbstractInputRecordProcessor

veniceRecordReader

Fields inherited from class com.linkedin.venice.hadoop.task.datawriter.AbstractDataWriterTask

TASK_ID_NOT_SET

Constructor Summary

Constructors

Constructor	Description
AbstractVeniceMapper()

Method Summary

Modifier and Type	Method	Description
void	configure(org.apache.hadoop.mapred.JobConf job)
void	map(INPUT_KEY inputKey, INPUT_VALUE inputValue, org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable> output, org.apache.hadoop.mapred.Reporter reporter)

Methods inherited from class com.linkedin.venice.hadoop.task.datawriter.AbstractInputRecordProcessor

close, configureTask, getRecordReader, process, processRecord, readDictionaryFromKafka

Methods inherited from class com.linkedin.venice.hadoop.task.datawriter.AbstractDataWriterTask

configure, getEngineTaskConfigProvider, getPartitionCount, getTaskId, isChunkingEnabled, isRmdChunkingEnabled, setChunkingEnabled

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.io.Closeable

close

Constructor Details

AbstractVeniceMapper

public AbstractVeniceMapper()

Method Details

map

public void map(INPUT_KEY inputKey,
 INPUT_VALUE inputValue,
 org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable> output,
 org.apache.hadoop.mapred.Reporter reporter)
         throws IOException

Specified by:

map in interface org.apache.hadoop.mapred.Mapper<INPUT_KEY,INPUT_VALUE,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

configure

public void configure(org.apache.hadoop.mapred.JobConf job)

Specified by:

configure in interface org.apache.hadoop.mapred.JobConfigurable