Package com.linkedin.venice.hadoop.input.recordreader.avro

Class AbstractAvroRecordReader<INPUT_KEY,INPUT_VALUE>

java.lang.Object

com.linkedin.venice.hadoop.input.recordreader.AbstractVeniceRecordReader<INPUT_KEY,INPUT_VALUE>

com.linkedin.venice.hadoop.input.recordreader.avro.AbstractAvroRecordReader<INPUT_KEY,INPUT_VALUE>

Type Parameters:

INPUT_KEY - The format of the key as controlled by the input format

INPUT_VALUE - The format of the value as controlled by the input format

Direct Known Subclasses:

VeniceAvroRecordReader

public abstract class AbstractAvroRecordReader<INPUT_KEY,INPUT_VALUE>
extends AbstractVeniceRecordReader<INPUT_KEY,INPUT_VALUE>

An abstraction for a record reader that reads records from input into Avro-serialized keys and values.

Constructor Summary

Constructors

Constructor	Description
AbstractAvroRecordReader(org.apache.avro.Schema dataSchema, java.lang.String keyFieldStr, java.lang.String valueFieldStr, ETLValueSchemaTransformation etlValueSchemaTransformation, org.apache.avro.Schema updateSchema)	This constructor is used when data is read from HDFS.

Method Summary

Modifier and Type	Method	Description
java.lang.Object	getAvroKey(INPUT_KEY inputKey, INPUT_VALUE inputValue)	Return an Avro output key
java.lang.Object	getAvroValue(INPUT_KEY inputKey, INPUT_VALUE inputValue)	return an Avro output value
org.apache.avro.Schema	getDataSchema()
protected abstract org.apache.avro.generic.IndexedRecord	getRecordDatum(INPUT_KEY inputKey, INPUT_VALUE inputValue)

Methods inherited from class com.linkedin.venice.hadoop.input.recordreader.AbstractVeniceRecordReader

configure, getKeyBytes, getKeySchema, getValueBytes, getValueSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractAvroRecordReader

public AbstractAvroRecordReader(org.apache.avro.Schema dataSchema,
                                java.lang.String keyFieldStr,
                                java.lang.String valueFieldStr,
                                ETLValueSchemaTransformation etlValueSchemaTransformation,
                                org.apache.avro.Schema updateSchema)

This constructor is used when data is read from HDFS.

Parameters:

dataSchema - Schema of the avro file

keyFieldStr - Field name of the key field

valueFieldStr - Field name of the value field

etlValueSchemaTransformation - The type of transformation that was applied to this schema during ETL. When source data set is not an ETL job, use NONE.

Method Detail

getDataSchema

public org.apache.avro.Schema getDataSchema()

getRecordDatum

protected abstract org.apache.avro.generic.IndexedRecord getRecordDatum(INPUT_KEY inputKey,
                                                                        INPUT_VALUE inputValue)

getAvroKey

public java.lang.Object getAvroKey(INPUT_KEY inputKey,
                                   INPUT_VALUE inputValue)

Description copied from class: AbstractVeniceRecordReader

Return an Avro output key

Specified by:

getAvroKey in class AbstractVeniceRecordReader<INPUT_KEY,INPUT_VALUE>

getAvroValue

public java.lang.Object getAvroValue(INPUT_KEY inputKey,
                                     INPUT_VALUE inputValue)

Description copied from class: AbstractVeniceRecordReader

return an Avro output value

Specified by:

getAvroValue in class AbstractVeniceRecordReader<INPUT_KEY,INPUT_VALUE>