Package com.linkedin.venice.hadoop.jobs

Class DataWriterComputeJob

java.lang.Object

com.linkedin.venice.hadoop.jobs.DataWriterComputeJob

All Implemented Interfaces:

ComputeJob, java.io.Closeable, java.lang.AutoCloseable

Direct Known Subclasses:

AbstractDataWriterSparkJob, DataWriterMRJob

public abstract class DataWriterComputeJob
extends java.lang.Object
implements ComputeJob

An abstraction for executing and monitoring a data writer compute job running on any batch compute engine

Nested Class Summary

Nested classes/interfaces inherited from interface com.linkedin.venice.hadoop.jobs.ComputeJob

ComputeJob.Status

Field Summary

Fields

Modifier and Type	Field	Description
static java.util.List<java.lang.String>	PASS_THROUGH_CONFIG_PREFIXES	Pass-through the properties whose names start with: VeniceWriter.VENICE_WRITER_CONFIG_PREFIX ConfigKeys.KAFKA_CONFIG_PREFIX KafkaInputRecordReader.KIF_RECORD_READER_KAFKA_CONFIG_PREFIX

Constructor Summary

Constructors

Constructor	Description
DataWriterComputeJob()

Method Summary

Modifier and Type	Method	Description
void	configure(VeniceProperties properties)
abstract void	configure(VeniceProperties props, PushJobSetting pushJobSetting)
java.lang.Throwable	getFailureReason()
protected abstract PushJobSetting	getPushJobSetting()
ComputeJob.Status	getStatus()
abstract DataWriterTaskTracker	getTaskTracker()
void	kill()
protected abstract void	runComputeJob()
void	runJob()
protected void	validateJob()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.venice.hadoop.jobs.ComputeJob

close

Field Detail

PASS_THROUGH_CONFIG_PREFIXES

public static final java.util.List<java.lang.String> PASS_THROUGH_CONFIG_PREFIXES

Pass-through the properties whose names start with:

• VeniceWriter.VENICE_WRITER_CONFIG_PREFIX
• ConfigKeys.KAFKA_CONFIG_PREFIX
• KafkaInputRecordReader.KIF_RECORD_READER_KAFKA_CONFIG_PREFIX

Constructor Detail

DataWriterComputeJob

public DataWriterComputeJob()

Method Detail

getTaskTracker

public abstract DataWriterTaskTracker getTaskTracker()

validateJob

protected void validateJob()

runComputeJob

protected abstract void runComputeJob()

configure

public void configure(VeniceProperties properties)

Specified by:

configure in interface ComputeJob

getPushJobSetting

protected abstract PushJobSetting getPushJobSetting()

configure

public abstract void configure(VeniceProperties props,
                               PushJobSetting pushJobSetting)

runJob

public void runJob()

Specified by:

runJob in interface ComputeJob

kill

public void kill()

Specified by:

kill in interface ComputeJob

getStatus

public ComputeJob.Status getStatus()

Specified by:

getStatus in interface ComputeJob

getFailureReason

public java.lang.Throwable getFailureReason()

Specified by:

getFailureReason in interface ComputeJob