Package com.linkedin.davinci.client
Class AvroGenericDaVinciClient<K,V>
java.lang.Object
com.linkedin.davinci.client.AvroGenericDaVinciClient<K,V>
- All Implemented Interfaces:
DaVinciClient<K,
,V> AvroGenericReadComputeStoreClient<K,
,V> AvroGenericStoreClient<K,
,V> Closeable
,AutoCloseable
- Direct Known Subclasses:
AvroSpecificDaVinciClient
public class AvroGenericDaVinciClient<K,V>
extends Object
implements DaVinciClient<K,V>, AvroGenericReadComputeStoreClient<K,V>
-
Field Summary
Modifier and TypeFieldDescriptionstatic final int
protected final org.apache.logging.log4j.Logger
static final ExecutorService
The following two fields are used to speed up the requests with a big number of keys: 1. -
Constructor Summary
ModifierConstructorDescriptionAvroGenericDaVinciClient
(DaVinciConfig daVinciConfig, ClientConfig clientConfig, VeniceProperties backendConfig, Optional<Set<String>> managedClients) protected
AvroGenericDaVinciClient
(DaVinciConfig daVinciConfig, ClientConfig clientConfig, VeniceProperties backendConfig, Optional<Set<String>> managedClients, ICProvider icProvider, AbstractAvroChunkingAdapter<V> chunkingAdapter, Runnable preValidation, Executor readChunkExecutorForLargeRequest) AvroGenericDaVinciClient
(DaVinciConfig daVinciConfig, ClientConfig clientConfig, VeniceProperties backendConfig, Optional<Set<String>> managedClients, ICProvider icProvider, Executor readChunkExecutorForLargeRequest) -
Method Summary
Modifier and TypeMethodDescriptionGet the values associated with the given keys and return them in a map of keys to values.void
close()
Release the internal resources.void
compute
(ComputeRequestWrapper computeRequestWrapper, Set<K> keys, org.apache.avro.Schema resultSchema, StreamingCallback<K, ComputeGenericRecord> callback, long preRequestTimeInNS) compute
(Optional<ClientStats> stats, AvroGenericReadComputeStoreClient computeStoreClient) void
computeWithKeyPrefixFilter
(byte[] keyPrefix, ComputeRequestWrapper computeRequestWrapper, StreamingCallback<org.apache.avro.generic.GenericRecord, org.apache.avro.generic.GenericRecord> callback) protected D2ServiceDiscoveryResponse
Lookup the value by given key, and get(key).get() will return null if it doesn't exist.Similar toAvroGenericStoreClient.get(Object)
except that it allows passing in a instance, to minimize GC.protected AbstractAvroChunkingAdapter<V>
static DaVinciBackend
protected GenericRecordChunkingAdapter
org.apache.avro.Schema
Get key schema.org.apache.avro.Schema
Deprecated.int
Get partition count of a store.protected void
initBackend
(ClientConfig clientConfig, VeniceConfigLoader configLoader, Optional<Set<String>> managedClients, ICProvider icProvider, Optional<ObjectCacheConfig> cacheConfig, DaVinciRecordTransformerConfig recordTransformerConfig) protected boolean
isPartitionReadyToServe
(VersionBackend versionBackend, int partition) Check if user partition is ready to serve traffic.protected boolean
isPartitionSubscribed
(VersionBackend versionBackend, int partition) boolean
boolean
isReady()
void
start()
void
streamingBatchGet
(Set<K> keys, StreamingCallback<K, V> callback) Streaming interface forAvroGenericStoreClient.batchGet(Set)
.protected CompletableFuture<Void>
subscribe
(ComplementSet<Integer> partitions) Ingest specific partition/partitions locally.Ingest the entire data (i.e.protected void
toString()
protected void
unsubscribe
(ComplementSet<Integer> partitions) void
unsubscribe
(Set<Integer> partitions) Stop ingesting a partition locally, and drop its associated local states/data.void
Stop ingesting all subscribed partition locally, and drop their associated local states/data.Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface com.linkedin.venice.client.store.AvroGenericReadComputeStoreClient
compute, compute, compute
Methods inherited from interface com.linkedin.venice.client.store.AvroGenericStoreClient
streamingBatchGet
-
Field Details
-
logger
protected final org.apache.logging.log4j.Logger logger -
READ_CHUNK_EXECUTOR
The following two fields are used to speed up the requests with a big number of keys: 1. Split the big request into smaller chunks. 2. Execute these chunks concurrently. -
DEFAULT_CHUNK_SPLIT_THRESHOLD
public static final int DEFAULT_CHUNK_SPLIT_THRESHOLD- See Also:
-
-
Constructor Details
-
AvroGenericDaVinciClient
public AvroGenericDaVinciClient(DaVinciConfig daVinciConfig, ClientConfig clientConfig, VeniceProperties backendConfig, Optional<Set<String>> managedClients) -
AvroGenericDaVinciClient
public AvroGenericDaVinciClient(DaVinciConfig daVinciConfig, ClientConfig clientConfig, VeniceProperties backendConfig, Optional<Set<String>> managedClients, ICProvider icProvider, Executor readChunkExecutorForLargeRequest) -
AvroGenericDaVinciClient
protected AvroGenericDaVinciClient(DaVinciConfig daVinciConfig, ClientConfig clientConfig, VeniceProperties backendConfig, Optional<Set<String>> managedClients, ICProvider icProvider, AbstractAvroChunkingAdapter<V> chunkingAdapter, Runnable preValidation, Executor readChunkExecutorForLargeRequest)
-
-
Method Details
-
getSchemaReader
- Specified by:
getSchemaReader
in interfaceAvroGenericReadComputeStoreClient<K,
V>
-
getStoreName
- Specified by:
getStoreName
in interfaceAvroGenericStoreClient<K,
V>
-
getKeySchema
public org.apache.avro.Schema getKeySchema()Description copied from interface:AvroGenericStoreClient
Get key schema.- Specified by:
getKeySchema
in interfaceAvroGenericStoreClient<K,
V>
-
getLatestValueSchema
Deprecated.Description copied from interface:AvroGenericStoreClient
Get the latest value schema known in current store client. This function doesn't guarantee it will return the latest schema if you add a new value schema when current store client is running.- Specified by:
getLatestValueSchema
in interfaceAvroGenericStoreClient<K,
V>
-
getPartitionCount
public int getPartitionCount()Description copied from interface:DaVinciClient
Get partition count of a store.- Specified by:
getPartitionCount
in interfaceDaVinciClient<K,
V> - Returns:
- partition count
-
subscribeAll
Description copied from interface:DaVinciClient
Ingest the entire data (i.e. all partitions) locally.- Specified by:
subscribeAll
in interfaceDaVinciClient<K,
V> - Returns:
- a future which completes when the data is ready to serve
-
subscribe
Description copied from interface:DaVinciClient
Ingest specific partition/partitions locally.- Specified by:
subscribe
in interfaceDaVinciClient<K,
V> - Parameters:
partitions
- the set of partition IDs to subscribe to- Returns:
- a future which completes when the partitions are ready to serve
-
subscribe
-
unsubscribeAll
public void unsubscribeAll()Description copied from interface:DaVinciClient
Stop ingesting all subscribed partition locally, and drop their associated local states/data. If applications intend to keep the states/data for future use, no need to invoke this function before callingAvroGenericStoreClient.close()
.- Specified by:
unsubscribeAll
in interfaceDaVinciClient<K,
V>
-
unsubscribe
Description copied from interface:DaVinciClient
Stop ingesting a partition locally, and drop its associated local states/data. If applications intend to keep the states/data for future use, no need to invoke this function before callingAvroGenericStoreClient.close()
.- Specified by:
unsubscribe
in interfaceDaVinciClient<K,
V> - Parameters:
partitions
- the set of partition IDs to unsubscribe from
-
unsubscribe
-
get
Description copied from interface:AvroGenericStoreClient
Lookup the value by given key, and get(key).get() will return null if it doesn't exist. For now, if any backend error/exception happens, get(Object key).get() will throwExecutionException
, which is a wrapper of the real exception.- Specified by:
get
in interfaceAvroGenericStoreClient<K,
V> - Returns:
-
get
Description copied from interface:AvroGenericStoreClient
Similar toAvroGenericStoreClient.get(Object)
except that it allows passing in a instance, to minimize GC.- Specified by:
get
in interfaceAvroGenericStoreClient<K,
V>
-
split
-
batchGet
Description copied from interface:AvroGenericStoreClient
Get the values associated with the given keys and return them in a map of keys to values. Note that the returned map will only contain entries for the keys which have a value associated with them.- Specified by:
batchGet
in interfaceAvroGenericStoreClient<K,
V> - Returns:
- Throws:
VeniceClientException
-
streamingBatchGet
public void streamingBatchGet(Set<K> keys, StreamingCallback<K, V> callback) throws VeniceClientExceptionDescription copied from interface:AvroGenericStoreClient
Streaming interface forAvroGenericStoreClient.batchGet(Set)
. You can find more info inStreamingCallback
.- Specified by:
streamingBatchGet
in interfaceAvroGenericStoreClient<K,
V> - Throws:
VeniceClientException
-
isProjectionFieldValidationEnabled
public boolean isProjectionFieldValidationEnabled()- Specified by:
isProjectionFieldValidationEnabled
in interfaceAvroGenericReadComputeStoreClient<K,
V>
-
compute
public ComputeRequestBuilder<K> compute(Optional<ClientStats> stats, AvroGenericReadComputeStoreClient computeStoreClient) throws VeniceClientException - Specified by:
compute
in interfaceAvroGenericReadComputeStoreClient<K,
V> - Throws:
VeniceClientException
-
compute
public void compute(ComputeRequestWrapper computeRequestWrapper, Set<K> keys, org.apache.avro.Schema resultSchema, StreamingCallback<K, ComputeGenericRecord> callback, long preRequestTimeInNS) throws VeniceClientException- Specified by:
compute
in interfaceAvroGenericReadComputeStoreClient<K,
V> - Throws:
VeniceClientException
-
computeWithKeyPrefixFilter
public void computeWithKeyPrefixFilter(byte[] keyPrefix, ComputeRequestWrapper computeRequestWrapper, StreamingCallback<org.apache.avro.generic.GenericRecord, org.apache.avro.generic.GenericRecord> callback) - Specified by:
computeWithKeyPrefixFilter
in interfaceAvroGenericReadComputeStoreClient<K,
V>
-
isReady
public boolean isReady() -
isPartitionReadyToServe
Check if user partition is ready to serve traffic. -
isPartitionSubscribed
-
throwIfNotReady
protected void throwIfNotReady() -
getAvroChunkingAdapter
-
getGenericRecordChunkingAdapter
-
discoverService
-
initBackend
protected void initBackend(ClientConfig clientConfig, VeniceConfigLoader configLoader, Optional<Set<String>> managedClients, ICProvider icProvider, Optional<ObjectCacheConfig> cacheConfig, DaVinciRecordTransformerConfig recordTransformerConfig) -
getBackend
-
start
public void start()- Specified by:
start
in interfaceAvroGenericStoreClient<K,
V>
-
close
public void close()Description copied from interface:AvroGenericStoreClient
Release the internal resources.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceAvroGenericStoreClient<K,
V> - Specified by:
close
in interfaceCloseable
-
toString
-