Package com.linkedin.venice.client.store

Class AbstractAvroStoreClient<K,V>

java.lang.Object

com.linkedin.venice.client.store.InternalAvroStoreClient<K,V>

com.linkedin.venice.client.store.AbstractAvroStoreClient<K,V>

All Implemented Interfaces:

AvroGenericReadComputeStoreClient<K,V>, AvroGenericStoreClient<K,V>, Closeable, AutoCloseable

Direct Known Subclasses:

AvroGenericStoreClientImpl, AvroSpecificStoreClientImpl

public abstract class AbstractAvroStoreClient<K,V>
extends InternalAvroStoreClient<K,V>

Field Summary

Fields

Modifier and Type	Field	Description
static final String	B64_FORMAT
protected RecordSerializer<ByteBuffer>	computeRequestClientKeySerializer
protected RecordSerializer<K>	keySerializer
protected RecordSerializer<ByteBuffer>	multiGetRequestSerializer
protected final boolean	needSchemaReader
static final String	TYPE_COMPUTE
static final String	TYPE_STORAGE

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	AbstractAvroStoreClient(TransportClient transportClient, boolean needSchemaReader, ClientConfig clientConfig)

Method Summary

Modifier and Type	Method	Description
void	close()	The behavior of READ apis will be non-deterministic after `close` function is called.
void	compute(ComputeRequestWrapper computeRequest, Set<K> keys, org.apache.avro.Schema resultSchema, StreamingCallback<K,ComputeGenericRecord> callback, long preRequestTimeInNS)
protected RecordSerializer<K>	createKeySerializer()	Clients using different protocols for deserialized data (e.g VSON, Proto, etc) can override this method to serialize the respective POJO to Avro bytes
CompletableFuture<V>	get(K key, Optional<ClientStats> stats, long preRequestTimeInNS)
protected final ClientConfig	getClientConfig()
protected String	getComputeRequestPath()
abstract RecordDeserializer<V>	getDataRecordDeserializer(int schemaId)
static Executor	getDefaultDeserializationExecutor()
Executor	getDeserializationExecutor()
org.apache.avro.Schema	getKeySchema()	Get key schema.
protected RecordSerializer<K>	getKeySerializerForRequest()	This function will try to initialize the store client at most once in a blocking fashion, and if the init fails, one async thread will be kicked off to init the store client periodically until the init succeeds.
protected RecordSerializer<K>	getKeySerializerWithoutRetry()
org.apache.avro.Schema	getLatestValueSchema()	Deprecated.
CompletableFuture<byte[]>	getRaw(String requestPath, Optional<ClientStats> stats, long preRequestTimeInNS)
protected Optional<org.apache.avro.Schema>	getReaderSchema()
String	getRequestPathByKey(K key)
SchemaReader	getSchemaReader()
protected abstract AbstractAvroStoreClient<K,V>	getStoreClientForSchemaReader()	To avoid cycle dependency, we need to initialize another store client for schema reader.
String	getStoreName()
protected TransportClient	getTransportClient()
protected static boolean	handleCallbackForEmptyKeySet(Collection<?> keys, StreamingCallback callback)
protected void	init()	During the initialization, we do the cluster discovery at first to find the real end point this client need to talk to, before initializing the serializer.
boolean	isProjectionFieldValidationEnabled()
void	setAsyncStoreInitSleepIntervalMs(long intervalMs)
void	start()
void	streamingBatchGet(Set<K> keys, StreamingCallback<K,V> callback)	Streaming interface for AvroGenericStoreClient.batchGet(Set).
String	toString()
static <T, K> T	tryToDeserializeWithVerboseLogging(RecordDeserializer<T> dataDeserializer, ByteBuffer data, int writerSchemaId, K key, RecordSerializer<K> keySerializer, SchemaReader schemaReader, org.apache.logging.log4j.Logger LOGGER)

Methods inherited from class com.linkedin.venice.client.store.InternalAvroStoreClient

computeWithKeyPrefixFilter, get, getRaw

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.linkedin.venice.client.store.AvroGenericReadComputeStoreClient

compute, compute, compute, compute

Methods inherited from interface com.linkedin.venice.client.store.AvroGenericStoreClient

batchGet, get, streamingBatchGet

Field Details

TYPE_STORAGE

public static final String TYPE_STORAGE

See Also:

• Constant Field Values

TYPE_COMPUTE

public static final String TYPE_COMPUTE

See Also:

• Constant Field Values

B64_FORMAT

public static final String B64_FORMAT

See Also:

• Constant Field Values

needSchemaReader

protected final boolean needSchemaReader

keySerializer

protected volatile RecordSerializer<K> keySerializer

multiGetRequestSerializer

protected RecordSerializer<ByteBuffer> multiGetRequestSerializer

computeRequestClientKeySerializer

protected RecordSerializer<ByteBuffer> computeRequestClientKeySerializer

Constructor Details

AbstractAvroStoreClient

protected AbstractAvroStoreClient(TransportClient transportClient,
 boolean needSchemaReader,
 ClientConfig clientConfig)

Method Details

getDefaultDeserializationExecutor

public static Executor getDefaultDeserializationExecutor()

getStoreName

public String getStoreName()

getClientConfig

protected final ClientConfig getClientConfig()

getTransportClient

protected TransportClient getTransportClient()

getSchemaReader

public SchemaReader getSchemaReader()

getDeserializationExecutor

public Executor getDeserializationExecutor()

Overrides:

getDeserializationExecutor in class InternalAvroStoreClient<K,V>

getComputeRequestPath

protected String getComputeRequestPath()

setAsyncStoreInitSleepIntervalMs

public void setAsyncStoreInitSleepIntervalMs(long intervalMs)

getKeySerializerForRequest

protected RecordSerializer<K> getKeySerializerForRequest()

This function will try to initialize the store client at most once in a blocking fashion, and if the init fails, one async thread will be kicked off to init the store client periodically until the init succeeds.

getKeySerializerWithoutRetry

protected RecordSerializer<K> getKeySerializerWithoutRetry()

init

protected void init()

During the initialization, we do the cluster discovery at first to find the real end point this client need to talk to, before initializing the serializer. So if sub-implementation needs to have its own serializer, please override the createKeySerializer method.

createKeySerializer

protected RecordSerializer<K> createKeySerializer()

Clients using different protocols for deserialized data (e.g VSON, Proto, etc) can override this method to serialize the respective POJO to Avro bytes

Returns:

A serializer for key objects to Avro bytes

getRequestPathByKey

public String getRequestPathByKey(K key)
                           throws VeniceClientException

Throws:

VeniceClientException

get

public CompletableFuture<V> get(K key,
 Optional<ClientStats> stats,
 long preRequestTimeInNS)
                         throws VeniceClientException

Specified by:

get in class InternalAvroStoreClient<K,V>

Throws:

VeniceClientException

getRaw

public CompletableFuture<byte[]> getRaw(String requestPath,
 Optional<ClientStats> stats,
 long preRequestTimeInNS)

Specified by:

getRaw in class InternalAvroStoreClient<K,V>

tryToDeserializeWithVerboseLogging

public static <T,
K> T tryToDeserializeWithVerboseLogging(RecordDeserializer<T> dataDeserializer,
 ByteBuffer data,
 int writerSchemaId,
 K key,
 RecordSerializer<K> keySerializer,
 SchemaReader schemaReader,
 org.apache.logging.log4j.Logger LOGGER)

isProjectionFieldValidationEnabled

public boolean isProjectionFieldValidationEnabled()

compute

public void compute(ComputeRequestWrapper computeRequest,
 Set<K> keys,
 org.apache.avro.Schema resultSchema,
 StreamingCallback<K,ComputeGenericRecord> callback,
 long preRequestTimeInNS)
             throws VeniceClientException

Throws:

VeniceClientException

start

public void start()
           throws VeniceClientException

Throws:

VeniceClientException

close

public void close()

The behavior of READ apis will be non-deterministic after `close` function is called.

getReaderSchema

protected Optional<org.apache.avro.Schema> getReaderSchema()

getStoreClientForSchemaReader

protected abstract AbstractAvroStoreClient<K,V> getStoreClientForSchemaReader()

To avoid cycle dependency, we need to initialize another store client for schema reader.

Returns:

Throws:

VeniceClientException

getDataRecordDeserializer

public abstract RecordDeserializer<V> getDataRecordDeserializer(int schemaId)
                                                         throws VeniceClientException

Throws:

VeniceClientException

toString

public String toString()

Overrides:

toString in class Object

getKeySchema

public org.apache.avro.Schema getKeySchema()

Description copied from interface: AvroGenericStoreClient

Get key schema.

getLatestValueSchema

@Deprecated
public org.apache.avro.Schema getLatestValueSchema()

Deprecated.

Description copied from interface: AvroGenericStoreClient

Get the latest value schema known in current store client. This function doesn't guarantee it will return the latest schema if you add a new value schema when current store client is running.

streamingBatchGet

public void streamingBatchGet(Set<K> keys,
 StreamingCallback<K,V> callback)
                       throws VeniceClientException

Description copied from interface: AvroGenericStoreClient

Streaming interface for AvroGenericStoreClient.batchGet(Set). You can find more info in StreamingCallback.

Throws:

VeniceClientException

handleCallbackForEmptyKeySet

protected static boolean handleCallbackForEmptyKeySet(Collection<?> keys,
 StreamingCallback callback)