Package com.linkedin.venice.hadoop.task.datawriter

Class ComplexVeniceWriterAdapter<K,V,U>

java.lang.Object

com.linkedin.venice.writer.AbstractVeniceWriter<K,V,U>

com.linkedin.venice.hadoop.task.datawriter.ComplexVeniceWriterAdapter<K,V,U>

All Implemented Interfaces:

Closeable, AutoCloseable

public class ComplexVeniceWriterAdapter<K,V,U>
extends AbstractVeniceWriter<K,V,U>

Adapter class for ComplexVeniceWriter to support public APIs defined in AbstractVeniceWriter in the context of being called in a CompositeVeniceWriter from VPJ. This class will provide capabilities to deserialize the value in order to provide ComplexVeniceWriter a value provider, and decompression capabilities in case of a re-push (Kafka input).

Field Summary

Fields inherited from class com.linkedin.venice.writer.AbstractVeniceWriter

topicName

Constructor Summary

Constructors

Constructor	Description
ComplexVeniceWriterAdapter(String topicName, ComplexVeniceWriter<K,V,U> veniceWriter, BiFunction<V,Integer,org.apache.avro.generic.GenericRecord> deserializeFunction, Function<V,V> decompressFunction)

Method Summary

Modifier and Type	Method	Description
void	close()
void	close(boolean gracefulClose)
CompletableFuture<PubSubProduceResult>	delete(K key, PubSubProducerCallback callback, DeleteMetadata deleteMetadata)	In VPJ, only re-push can trigger this function.
void	flush()
CompletableFuture<PubSubProduceResult>	put(K key, V value, int valueSchemaId, PubSubProducerCallback callback)
CompletableFuture<PubSubProduceResult>	put(K key, V value, int valueSchemaId, PubSubProducerCallback callback, PutMetadata putMetadata)	The PubSubProduceResult will always be null and should not be used.
Future<PubSubProduceResult>	update(K key, U update, int valueSchemaId, int derivedSchemaId, PubSubProducerCallback callback)

Methods inherited from class com.linkedin.venice.writer.AbstractVeniceWriter

getTopicName, put

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ComplexVeniceWriterAdapter

public ComplexVeniceWriterAdapter(String topicName,
 ComplexVeniceWriter<K,V,U> veniceWriter,
 BiFunction<V,Integer,org.apache.avro.generic.GenericRecord> deserializeFunction,
 Function<V,V> decompressFunction)

Method Details

close

public void close(boolean gracefulClose)
           throws IOException

Specified by:

close in class AbstractVeniceWriter<K,V,U>

Throws:

IOException

put

public CompletableFuture<PubSubProduceResult> put(K key,
 V value,
 int valueSchemaId,
 PubSubProducerCallback callback)

Specified by:

put in class AbstractVeniceWriter<K,V,U>

put

public CompletableFuture<PubSubProduceResult> put(K key,
 V value,
 int valueSchemaId,
 PubSubProducerCallback callback,
 PutMetadata putMetadata)

The PubSubProduceResult will always be null and should not be used. This is acceptable because: 1. ComplexVeniceWriter.complexPut(Object, Object, int, Lazy) returns a CompletableFuture with Void since it could potentially write to multiple partitions resulting in multiple PubSubProduceResult. 2. Only the PubSubProduceResult of the main writer in CompositeVeniceWriter is used for reporting purpose in VPJ.

Specified by:

put in class AbstractVeniceWriter<K,V,U>

delete

public CompletableFuture<PubSubProduceResult> delete(K key,
 PubSubProducerCallback callback,
 DeleteMetadata deleteMetadata)

In VPJ, only re-push can trigger this function. During re-push the deletion to view topics are useless and should be ignored.

Specified by:

delete in class AbstractVeniceWriter<K,V,U>

update

public Future<PubSubProduceResult> update(K key,
 U update,
 int valueSchemaId,
 int derivedSchemaId,
 PubSubProducerCallback callback)

Specified by:

update in class AbstractVeniceWriter<K,V,U>

flush

public void flush()

Specified by:

flush in class AbstractVeniceWriter<K,V,U>

close

public void close()
           throws IOException

Throws:

IOException