Package com.linkedin.venice.hadoop.input.recordreader.avro

Class VeniceAvroRecordReader

java.lang.Object

com.linkedin.venice.hadoop.input.recordreader.AbstractVeniceRecordReader<INPUT_KEY,INPUT_VALUE>

com.linkedin.venice.hadoop.input.recordreader.avro.AbstractAvroRecordReader<org.apache.avro.mapred.AvroWrapper<org.apache.avro.generic.IndexedRecord>,org.apache.hadoop.io.NullWritable>

com.linkedin.venice.hadoop.input.recordreader.avro.VeniceAvroRecordReader

public class VeniceAvroRecordReader
extends AbstractAvroRecordReader<org.apache.avro.mapred.AvroWrapper<org.apache.avro.generic.IndexedRecord>,org.apache.hadoop.io.NullWritable>

A record reader that reads records from Avro file input into Avro-serialized keys and values.

Constructor Summary

Constructors

Constructor	Description
VeniceAvroRecordReader(org.apache.avro.Schema dataSchema, String keyFieldStr, String valueFieldStr, String rmdFieldStr, ETLValueSchemaTransformation etlValueSchemaTransformation, org.apache.avro.Schema updateSchema)	This constructor is used when data is read from HDFS.
VeniceAvroRecordReader(org.apache.avro.Schema dataSchema, String keyFieldStr, String valueFieldStr, String rmdFieldStr, ETLValueSchemaTransformation etlValueSchemaTransformation, org.apache.avro.Schema updateSchema, org.apache.avro.Schema writerValueSchema, org.apache.avro.Schema writerRmdSchema)

Method Summary

Modifier and Type	Method	Description
static VeniceAvroRecordReader	fromProps(VeniceProperties props)
protected org.apache.avro.generic.IndexedRecord	getRecordDatum(org.apache.avro.mapred.AvroWrapper<org.apache.avro.generic.IndexedRecord> record, org.apache.hadoop.io.NullWritable nullValue)

Methods inherited from class com.linkedin.venice.hadoop.input.recordreader.avro.AbstractAvroRecordReader

getAvroKey, getAvroValue, getDataSchema, getRmdValue

Methods inherited from class com.linkedin.venice.hadoop.input.recordreader.AbstractVeniceRecordReader

configure, configure, getKeyBytes, getKeySchema, getRmdBytes, getRmdSchema, getValueBytes, getValueSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VeniceAvroRecordReader

public VeniceAvroRecordReader(org.apache.avro.Schema dataSchema,
 String keyFieldStr,
 String valueFieldStr,
 String rmdFieldStr,
 ETLValueSchemaTransformation etlValueSchemaTransformation,
 org.apache.avro.Schema updateSchema)

This constructor is used when data is read from HDFS.

Parameters:

dataSchema - Schema of the avro file

keyFieldStr - Field name of the key field

valueFieldStr - Field name of the value field

etlValueSchemaTransformation - The type of transformation that was applied to this schema during ETL. When source data set is not an ETL job, use NONE.

VeniceAvroRecordReader

public VeniceAvroRecordReader(org.apache.avro.Schema dataSchema,
 String keyFieldStr,
 String valueFieldStr,
 String rmdFieldStr,
 ETLValueSchemaTransformation etlValueSchemaTransformation,
 org.apache.avro.Schema updateSchema,
 org.apache.avro.Schema writerValueSchema,
 org.apache.avro.Schema writerRmdSchema)

Method Details

fromProps

public static VeniceAvroRecordReader fromProps(VeniceProperties props)

getRecordDatum

protected org.apache.avro.generic.IndexedRecord getRecordDatum(org.apache.avro.mapred.AvroWrapper<org.apache.avro.generic.IndexedRecord> record,
 org.apache.hadoop.io.NullWritable nullValue)

Specified by:

getRecordDatum in class AbstractAvroRecordReader<org.apache.avro.mapred.AvroWrapper<org.apache.avro.generic.IndexedRecord>,org.apache.hadoop.io.NullWritable>