Package com.linkedin.venice.compute

Class ComputeUtils

java.lang.Object

com.linkedin.venice.compute.ComputeUtils

public class ComputeUtils
extends Object

This class provides utilities for float-vector operations, and it also handles PrimitiveFloatList transparently to the user of this class.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	CACHED_SQUARED_L2_NORM_KEY
static final String	ILLEGAL_AVRO_CHARACTER
static final String	ILLEGAL_AVRO_CHARACTER_REPLACEMENT
static final Pattern	VALID_AVRO_NAME_PATTERN

Constructor Summary

Constructors

Constructor	Description
ComputeUtils()

Method Summary

Modifier and Type	Method	Description
static void	checkResultSchema(org.apache.avro.Schema resultSchema, org.apache.avro.Schema valueSchema, List<ComputeOperation> operations)
static org.apache.avro.generic.GenericRecord	computeResult(List<ComputeOperation> operations, List<org.apache.avro.Schema.Field> operationResultFields, Map<String,Object> sharedContext, org.apache.avro.generic.GenericRecord inputRecord, org.apache.avro.generic.GenericRecord outputRecord)
static org.apache.avro.generic.GenericRecord	computeResult(List<ComputeOperation> operations, List<org.apache.avro.Schema.Field> operationResultFields, Map<String,Object> sharedContext, org.apache.avro.generic.GenericRecord inputRecord, org.apache.avro.Schema outputSchema)
static ComputeRequest	deserializeComputeRequest(org.apache.avro.io.BinaryDecoder decoder, ComputeRequest reuse)
static float	dotProduct(List<Float> list1, List<Float> list2)
static <T> List<T>	getNullableFieldValueAsList(org.apache.avro.generic.GenericRecord record, org.apache.avro.Schema.Field field)
static List<org.apache.avro.Schema.Field>	getOperationResultFields(List<ComputeOperation> operations, org.apache.avro.Schema resultSchema)
static List<Float>	hadamardProduct(List<Float> list1, List<Float> list2)
static String	removeAvroIllegalCharacter(String name)	According to Avro specification (https://avro.apache.org/docs/1.7.7/spec.html#Names): The name portion of a fullname, record field names, and enum symbols must: 1.
static float	squaredL2Norm(List<Float> list)
static String	validateNullableFieldAndGetErrorMsg(ReadComputeOperator operator, org.apache.avro.generic.GenericRecord valueRecord, org.apache.avro.Schema.Field operatorField, String operatorFieldName)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

CACHED_SQUARED_L2_NORM_KEY

public static final String CACHED_SQUARED_L2_NORM_KEY

See Also:

• Constant Field Values

VALID_AVRO_NAME_PATTERN

public static final Pattern VALID_AVRO_NAME_PATTERN

ILLEGAL_AVRO_CHARACTER

public static final String ILLEGAL_AVRO_CHARACTER

See Also:

• Constant Field Values

ILLEGAL_AVRO_CHARACTER_REPLACEMENT

public static final String ILLEGAL_AVRO_CHARACTER_REPLACEMENT

See Also:

• Constant Field Values

Constructor Details

ComputeUtils

public ComputeUtils()

Method Details

deserializeComputeRequest

public static ComputeRequest deserializeComputeRequest(org.apache.avro.io.BinaryDecoder decoder,
 ComputeRequest reuse)

checkResultSchema

public static void checkResultSchema(org.apache.avro.Schema resultSchema,
 org.apache.avro.Schema valueSchema,
 List<ComputeOperation> operations)

removeAvroIllegalCharacter

public static String removeAvroIllegalCharacter(String name)

According to Avro specification (https://avro.apache.org/docs/1.7.7/spec.html#Names): The name portion of a fullname, record field names, and enum symbols must: 1. start with [A-Za-z_] 2. subsequently contain only [A-Za-z0-9_] Remove all Avro illegal characters.

Parameters:

name -

Returns:

a string that doesn't contain any illegal character

dotProduct

public static float dotProduct(List<Float> list1,
 List<Float> list2)

hadamardProduct

public static List<Float> hadamardProduct(List<Float> list1,
 List<Float> list2)

getOperationResultFields

public static List<org.apache.avro.Schema.Field> getOperationResultFields(List<ComputeOperation> operations,
 org.apache.avro.Schema resultSchema)

squaredL2Norm

public static float squaredL2Norm(List<Float> list)

getNullableFieldValueAsList

public static <T> List<T> getNullableFieldValueAsList(org.apache.avro.generic.GenericRecord record,
 org.apache.avro.Schema.Field field)

Type Parameters:

T - Type of the list element to cast the extracted value to

Parameters:

record - the record from which the value of the given field is extracted

field - field which is used to extract the value from the given record

Returns:

An unmodifiable empty list if the extracted value is null. Otherwise return a list that may or may not be modifiable depending on specified type of the list as a field in the record.

validateNullableFieldAndGetErrorMsg

public static String validateNullableFieldAndGetErrorMsg(ReadComputeOperator operator,
 org.apache.avro.generic.GenericRecord valueRecord,
 org.apache.avro.Schema.Field operatorField,
 String operatorFieldName)

Returns:

Error message if the nullable field validation failed or null otherwise.

computeResult

public static org.apache.avro.generic.GenericRecord computeResult(List<ComputeOperation> operations,
 List<org.apache.avro.Schema.Field> operationResultFields,
 Map<String,Object> sharedContext,
 org.apache.avro.generic.GenericRecord inputRecord,
 org.apache.avro.Schema outputSchema)

computeResult

public static org.apache.avro.generic.GenericRecord computeResult(List<ComputeOperation> operations,
 List<org.apache.avro.Schema.Field> operationResultFields,
 Map<String,Object> sharedContext,
 org.apache.avro.generic.GenericRecord inputRecord,
 org.apache.avro.generic.GenericRecord outputRecord)