Package com.linkedin.venice.compute

Class ComputeUtils

  • public class ComputeUtils
extends java.lang.Object
    This class provides utilities for float-vector operations, and it also handles PrimitiveFloatList transparently to the user of this class.

    • Constructor Summary

      Constructors 
      Constructor Description
      ComputeUtils()  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static void checkResultSchema​(org.apache.avro.Schema resultSchema, org.apache.avro.Schema valueSchema, java.util.List<ComputeOperation> operations)  
      static org.apache.avro.generic.GenericRecord computeResult​(java.util.List<ComputeOperation> operations, java.util.List<org.apache.avro.Schema.Field> operationResultFields, java.util.Map<java.lang.String,​java.lang.Object> sharedContext, org.apache.avro.generic.GenericRecord inputRecord, org.apache.avro.generic.GenericRecord outputRecord)  
      static org.apache.avro.generic.GenericRecord computeResult​(java.util.List<ComputeOperation> operations, java.util.List<org.apache.avro.Schema.Field> operationResultFields, java.util.Map<java.lang.String,​java.lang.Object> sharedContext, org.apache.avro.generic.GenericRecord inputRecord, org.apache.avro.Schema outputSchema)  
      static ComputeRequest deserializeComputeRequest​(org.apache.avro.io.BinaryDecoder decoder, ComputeRequest reuse)  
      static float dotProduct​(java.util.List<java.lang.Float> list1, java.util.List<java.lang.Float> list2)  
      static <T> java.util.List<T> getNullableFieldValueAsList​(org.apache.avro.generic.GenericRecord record, org.apache.avro.Schema.Field field)  
      static java.util.List<org.apache.avro.Schema.Field> getOperationResultFields​(java.util.List<ComputeOperation> operations, org.apache.avro.Schema resultSchema)  
      static java.util.List<java.lang.Float> hadamardProduct​(java.util.List<java.lang.Float> list1, java.util.List<java.lang.Float> list2)  
      static java.lang.String removeAvroIllegalCharacter​(java.lang.String name)
      According to Avro specification (https://avro.apache.org/docs/1.7.7/spec.html#Names): The name portion of a fullname, record field names, and enum symbols must: 1.
      static float squaredL2Norm​(java.util.List<java.lang.Float> list)  
      static java.lang.String validateNullableFieldAndGetErrorMsg​(ReadComputeOperator operator, org.apache.avro.generic.GenericRecord valueRecord, org.apache.avro.Schema.Field operatorField, java.lang.String operatorFieldName)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • CACHED_SQUARED_L2_NORM_KEY

        public static final java.lang.String CACHED_SQUARED_L2_NORM_KEY
        See Also:
        Constant Field Values

      • VALID_AVRO_NAME_PATTERN

        public static final java.util.regex.Pattern VALID_AVRO_NAME_PATTERN

      • ILLEGAL_AVRO_CHARACTER

        public static final java.lang.String ILLEGAL_AVRO_CHARACTER
        See Also:
        Constant Field Values

      • ILLEGAL_AVRO_CHARACTER_REPLACEMENT

        public static final java.lang.String ILLEGAL_AVRO_CHARACTER_REPLACEMENT
        See Also:
        Constant Field Values

    • Constructor Detail

      • ComputeUtils

        public ComputeUtils()

    • Method Detail

      • deserializeComputeRequest

        public static ComputeRequest deserializeComputeRequest​(org.apache.avro.io.BinaryDecoder decoder,
                                                       ComputeRequest reuse)

      • checkResultSchema

        public static void checkResultSchema​(org.apache.avro.Schema resultSchema,
                                     org.apache.avro.Schema valueSchema,
                                     java.util.List<ComputeOperation> operations)

      • removeAvroIllegalCharacter

        public static java.lang.String removeAvroIllegalCharacter​(java.lang.String name)
        According to Avro specification (https://avro.apache.org/docs/1.7.7/spec.html#Names): The name portion of a fullname, record field names, and enum symbols must: 1. start with [A-Za-z_] 2. subsequently contain only [A-Za-z0-9_] Remove all Avro illegal characters.
        Parameters:
        name -
        Returns:
        a string that doesn't contain any illegal character

      • dotProduct

        public static float dotProduct​(java.util.List<java.lang.Float> list1,
                               java.util.List<java.lang.Float> list2)

      • hadamardProduct

        public static java.util.List<java.lang.Float> hadamardProduct​(java.util.List<java.lang.Float> list1,
                                                              java.util.List<java.lang.Float> list2)

      • getOperationResultFields

        public static java.util.List<org.apache.avro.Schema.Field> getOperationResultFields​(java.util.List<ComputeOperation> operations,
                                                                                    org.apache.avro.Schema resultSchema)

      • squaredL2Norm

        public static float squaredL2Norm​(java.util.List<java.lang.Float> list)

      • getNullableFieldValueAsList

        public static <T> java.util.List<T> getNullableFieldValueAsList​(org.apache.avro.generic.GenericRecord record,
                                                                org.apache.avro.Schema.Field field)
        Type Parameters:
        T - Type of the list element to cast the extracted value to
        Parameters:
        record - the record from which the value of the given field is extracted
        field - field which is used to extract the value from the given record
        Returns:
        An unmodifiable empty list if the extracted value is null. Otherwise return a list that may or may not be modifiable depending on specified type of the list as a field in the record.

      • validateNullableFieldAndGetErrorMsg

        public static java.lang.String validateNullableFieldAndGetErrorMsg​(ReadComputeOperator operator,
                                                                   org.apache.avro.generic.GenericRecord valueRecord,
                                                                   org.apache.avro.Schema.Field operatorField,
                                                                   java.lang.String operatorFieldName)
        Returns:
        Error message if the nullable field validation failed or null otherwise.

      • computeResult

        public static org.apache.avro.generic.GenericRecord computeResult​(java.util.List<ComputeOperation> operations,
                                                                  java.util.List<org.apache.avro.Schema.Field> operationResultFields,
                                                                  java.util.Map<java.lang.String,​java.lang.Object> sharedContext,
                                                                  org.apache.avro.generic.GenericRecord inputRecord,
                                                                  org.apache.avro.Schema outputSchema)

      • computeResult

        public static org.apache.avro.generic.GenericRecord computeResult​(java.util.List<ComputeOperation> operations,
                                                                  java.util.List<org.apache.avro.Schema.Field> operationResultFields,
                                                                  java.util.Map<java.lang.String,​java.lang.Object> sharedContext,
                                                                  org.apache.avro.generic.GenericRecord inputRecord,
                                                                  org.apache.avro.generic.GenericRecord outputRecord)