Package com.linkedin.venice.client.store

Class AbstractAvroComputeRequestBuilder<K>

java.lang.Object

com.linkedin.venice.client.store.AbstractAvroComputeRequestBuilder<K>

Type Parameters:

K -

All Implemented Interfaces:

ComputeRequestBuilder<K>

Direct Known Subclasses:

AvroComputeRequestBuilderV3

public abstract class AbstractAvroComputeRequestBuilder<K>
extends Object
implements ComputeRequestBuilder<K>

This abstract class contains all the common field and APIs for compute request builder; for each new compute request version, there will be a new builder class that extends this class and the new builder will include customized fields and APIs.

Field Summary

Fields

Modifier and Type	Field	Description
protected static final org.apache.avro.Schema	COSINE_SIMILARITY_RESULT_SCHEMA
protected static final String	COSINE_SIMILARITY_SPEC
protected static final org.apache.avro.Schema	DOT_PRODUCT_RESULT_SCHEMA
protected static final String	DOT_PRODUCT_SPEC
protected static final org.apache.avro.Schema	HADAMARD_PRODUCT_RESULT_SCHEMA
protected static final String	HADAMARD_PRODUCT_SPEC
protected final org.apache.avro.Schema	latestValueSchema
protected final int	latestValueSchemaId
protected static final String	PROJECTION_SPEC
protected static final Map<Map<String,Object>,SchemaAndToString>	RESULT_SCHEMA_CACHE
protected final String	resultSchemaName
protected final AvroGenericReadComputeStoreClient	storeClient

Constructor Summary

Constructors

Constructor	Description
AbstractAvroComputeRequestBuilder(AvroGenericReadComputeStoreClient storeClient, SchemaReader schemaReader)

Method Summary

Modifier and Type	Method	Description
protected void	checkComputeFieldValidity(String computeFieldName, String resultFieldName, Set<String> resultFieldsSet, ComputeOperationType computeType)
protected Set<String>	commonValidityCheck()	Compute operation validity check for projections, dot-product and cosine-similarity.
ComputeRequestBuilder<K>	cosineSimilarity(String inputFieldName, List<Float> cosSimilarityParam, String resultFieldName)	Setup cosine-similarity operation.
ComputeRequestBuilder<K>	dotProduct(String inputFieldName, List<Float> dotProductParam, String resultFieldName)	Setup dot-product operation.
CompletableFuture<Map<K,ComputeGenericRecord>>	execute(Set<K> keys)	Send compute request to Venice, and this should be the last step of the compute specification.
protected ComputeRequestWrapper	generateComputeRequest(SchemaAndToString resultSchema, boolean originallyStreaming)
protected Map<String,Object>	getCommonComputeSpec()	Generate compute spec for projections, dot-product and cosine-similarity.
protected List<org.apache.avro.Schema.Field>	getCommonResultFields()	Generate compute result schema fields for projections, dot-product and cosine-similarity.
protected List<ComputeOperation>	getComputeRequestOperations()	Generate compute operations for projections, dot-product and cosine-similarity.
protected SchemaAndToString	getResultSchema()
ComputeRequestBuilder<K>	hadamardProduct(String inputFieldName, List<Float> hadamardProductParam, String resultFieldName)	Setup hadamard-product operation; if this api is invoked, use version 2 in the compute request version header.
ComputeRequestBuilder<K>	project(String... fieldNames)	Setup project fields, and right now only top-level fields are supported.
ComputeRequestBuilder<K>	project(Collection<String> fieldNames)	Setup project fields, and right now only top-level fields are supported.
AbstractAvroComputeRequestBuilder<K>	setStats(Optional<ClientStats> streamingStats)
AbstractAvroComputeRequestBuilder<K>	setTime(io.tehuti.utils.Time time)	test-only
AbstractAvroComputeRequestBuilder<K>	setValidateProjectionFields(boolean projectionFieldValidation)
CompletableFuture<VeniceResponseMap<K,ComputeGenericRecord>>	streamingExecute(Set<K> keys)	Send compute request to Venice, and this should be the last step of the compute specification.
void	streamingExecute(Set<K> keys, StreamingCallback<K,ComputeGenericRecord> callback)	Streaming interface for ComputeRequestBuilder.execute(Set), and you could find more info in StreamingCallback.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.venice.client.store.ComputeRequestBuilder

count, executeWithFilter

Field Details

RESULT_SCHEMA_CACHE

protected static final Map<Map<String,Object>,SchemaAndToString> RESULT_SCHEMA_CACHE

PROJECTION_SPEC

protected static final String PROJECTION_SPEC

See Also:

• Constant Field Values

DOT_PRODUCT_SPEC

protected static final String DOT_PRODUCT_SPEC

See Also:

• Constant Field Values

COSINE_SIMILARITY_SPEC

protected static final String COSINE_SIMILARITY_SPEC

See Also:

• Constant Field Values

HADAMARD_PRODUCT_SPEC

protected static final String HADAMARD_PRODUCT_SPEC

See Also:

• Constant Field Values

HADAMARD_PRODUCT_RESULT_SCHEMA

protected static final org.apache.avro.Schema HADAMARD_PRODUCT_RESULT_SCHEMA

DOT_PRODUCT_RESULT_SCHEMA

protected static final org.apache.avro.Schema DOT_PRODUCT_RESULT_SCHEMA

COSINE_SIMILARITY_RESULT_SCHEMA

protected static final org.apache.avro.Schema COSINE_SIMILARITY_RESULT_SCHEMA

storeClient

protected final AvroGenericReadComputeStoreClient storeClient

latestValueSchemaId

protected final int latestValueSchemaId

latestValueSchema

protected final org.apache.avro.Schema latestValueSchema

resultSchemaName

protected final String resultSchemaName

Constructor Details

AbstractAvroComputeRequestBuilder

public AbstractAvroComputeRequestBuilder(AvroGenericReadComputeStoreClient storeClient,
 SchemaReader schemaReader)

Method Details

setTime

public AbstractAvroComputeRequestBuilder<K> setTime(io.tehuti.utils.Time time)

test-only

setStats

public AbstractAvroComputeRequestBuilder<K> setStats(Optional<ClientStats> streamingStats)

setValidateProjectionFields

public AbstractAvroComputeRequestBuilder<K> setValidateProjectionFields(boolean projectionFieldValidation)

project

public ComputeRequestBuilder<K> project(String... fieldNames)
                                 throws VeniceClientException

Description copied from interface: ComputeRequestBuilder

Setup project fields, and right now only top-level fields are supported.

Specified by:

project in interface ComputeRequestBuilder<K>

Returns:

Throws:

VeniceClientException

project

public ComputeRequestBuilder<K> project(Collection<String> fieldNames)
                                 throws VeniceClientException

Description copied from interface: ComputeRequestBuilder

Setup project fields, and right now only top-level fields are supported.

Specified by:

project in interface ComputeRequestBuilder<K>

Returns:

Throws:

VeniceClientException

dotProduct

public ComputeRequestBuilder<K> dotProduct(String inputFieldName,
 List<Float> dotProductParam,
 String resultFieldName)
                                    throws VeniceClientException

Description copied from interface: ComputeRequestBuilder

Setup dot-product operation.

Specified by:

dotProduct in interface ComputeRequestBuilder<K>

Parameters:

inputFieldName - : top-level field in the value record as the input of dot-product operation

dotProductParam - : dot-product param

resultFieldName - : result field name in the response record

Returns:

Throws:

VeniceClientException

cosineSimilarity

public ComputeRequestBuilder<K> cosineSimilarity(String inputFieldName,
 List<Float> cosSimilarityParam,
 String resultFieldName)
                                          throws VeniceClientException

Description copied from interface: ComputeRequestBuilder

Setup cosine-similarity operation.

Specified by:

cosineSimilarity in interface ComputeRequestBuilder<K>

Parameters:

inputFieldName - : top-level field in the value record as the input of cosine-similarity operation

cosSimilarityParam - : cosine-similarity param

resultFieldName - : result field name in the response record

Returns:

Throws:

VeniceClientException

getCommonComputeSpec

protected Map<String,Object> getCommonComputeSpec()

Generate compute spec for projections, dot-product and cosine-similarity.

Returns:

common compute spec

commonValidityCheck

protected Set<String> commonValidityCheck()

Compute operation validity check for projections, dot-product and cosine-similarity.

Returns:

a set of existing operations result field name

getCommonResultFields

protected List<org.apache.avro.Schema.Field> getCommonResultFields()

Generate compute result schema fields for projections, dot-product and cosine-similarity.

Returns:

a list of existing compute result schema fields

getComputeRequestOperations

protected List<ComputeOperation> getComputeRequestOperations()

Generate compute operations for projections, dot-product and cosine-similarity.

Returns:

a list of existing compute operations

execute

public CompletableFuture<Map<K,ComputeGenericRecord>> execute(Set<K> keys)
                                                       throws VeniceClientException

Description copied from interface: ComputeRequestBuilder

Send compute request to Venice, and this should be the last step of the compute specification.

Specified by:

execute in interface ComputeRequestBuilder<K>

Parameters:

keys - : keys for the candidate records

Returns:

Throws:

VeniceClientException

streamingExecute

public CompletableFuture<VeniceResponseMap<K,ComputeGenericRecord>> streamingExecute(Set<K> keys)

Description copied from interface: ComputeRequestBuilder

Send compute request to Venice, and this should be the last step of the compute specification. The difference between this function and the previous ComputeRequestBuilder.execute(Set) is that this function will return the available response instead of throwing a TimeoutException when timeout happens: streamingExecute(keys).get(timeout, units);

Specified by:

streamingExecute in interface ComputeRequestBuilder<K>

Returns:

streamingExecute

public void streamingExecute(Set<K> keys,
 StreamingCallback<K,ComputeGenericRecord> callback)
                      throws VeniceClientException

Description copied from interface: ComputeRequestBuilder

Streaming interface for ComputeRequestBuilder.execute(Set), and you could find more info in StreamingCallback.

Specified by:

streamingExecute in interface ComputeRequestBuilder<K>

Throws:

VeniceClientException

checkComputeFieldValidity

protected void checkComputeFieldValidity(String computeFieldName,
 String resultFieldName,
 Set<String> resultFieldsSet,
 ComputeOperationType computeType)

getResultSchema

protected SchemaAndToString getResultSchema()

hadamardProduct

public ComputeRequestBuilder<K> hadamardProduct(String inputFieldName,
 List<Float> hadamardProductParam,
 String resultFieldName)
                                         throws VeniceClientException

Description copied from interface: ComputeRequestBuilder

Setup hadamard-product operation; if this api is invoked, use version 2 in the compute request version header.

Specified by:

hadamardProduct in interface ComputeRequestBuilder<K>

Parameters:

inputFieldName - : top-level field in the value record as the input of hadamard-product operation

hadamardProductParam - : hadamard-product param

resultFieldName - : result field name in the response record

Returns:

Throws:

VeniceClientException

generateComputeRequest

protected ComputeRequestWrapper generateComputeRequest(SchemaAndToString resultSchema,
 boolean originallyStreaming)