Package com.linkedin.venice.router.api.path

Class VeniceMultiKeyPath<K>

java.lang.Object

com.linkedin.venice.router.api.path.VenicePath

com.linkedin.venice.router.api.path.VeniceMultiKeyPath<K>

Type Parameters:

K - the key wrapper to be sent to servers for the given type of query

All Implemented Interfaces:

ResourcePath<RouterKey>

Direct Known Subclasses:

VeniceComputePath, VeniceMultiGetPath

public abstract class VeniceMultiKeyPath<K>
extends VenicePath

Multi-key requests support many additional functionalities compared to single gets, including: - Smart long-tail retry configurations - Streaming responses - Helix-assisted routing (HAR)

Field Summary

Fields

Modifier and Type	Field	Description
protected final Map<RouterKey,K>	routerKeyMap

Fields inherited from class com.linkedin.venice.router.api.path.VenicePath

retryConfig, retryManager, storeVersionName

Constructor Summary

Constructors

Constructor	Description
VeniceMultiKeyPath(StoreVersionName storeVersionName, RouterRetryConfig retryConfig, RetryManager retryManager, VeniceResponseDecompressor responseDecompressor)
VeniceMultiKeyPath(StoreVersionName storeVersionName, Map<RouterKey,K> routerKeyMap, RouterRetryConfig retryConfig, RetryManager retryManager, VeniceResponseDecompressor responseDecompressor)

Method Summary

Modifier and Type	Method	Description
org.apache.http.client.methods.HttpUriRequest	composeRouterRequestInternal(String storageNodeUri)
protected abstract K	createRouterRequestKey(ByteBuffer key, int keyIdx, int partitionId)	Create a router request key.
protected abstract VenicePath	fixRetryRequestForSubPath(Map<RouterKey,K> routerKeyMap)
byte[]	getBody()
VeniceChunkedResponse	getChunkedResponse()
int	getHelixGroupId()
io.netty.handler.codec.http.HttpMethod	getHttpMethod()
int	getLongTailRetryMaxRouteForMultiKeyReq()
int	getLongTailRetryThresholdMs()
long	getRequestId()
int	getSmartLongTailRetryAbortThresholdMs()
void	initialize(String resourceName, List<ByteBuffer> keys, VenicePartitionFinder partitionFinder, int maxKeyCount, AggRouterHttpRequestStats stats)	Fill the router key map and the index2routerKey map.
boolean	isLongTailRetryAllowedForNewRequest()
boolean	isSmartLongTailRetryEnabled()
protected abstract byte[]	serializeRouterRequest()	For multi-get requests, simply serialize the set of RouterKey to bytes; for read compute requests, concatenate the compute request and the serialized keys
void	setChunkedWriteHandler(io.netty.channel.ChannelHandlerContext ctx, VeniceChunkedWriteHandler chunkedWriteHandler, RouterStats<AggRouterHttpRequestStats> routerStats)
void	setHelixGroupId(int helixGroupId)
protected void	setupRetryRelatedInfo(VenicePath originalPath)
VenicePath	substitutePartitionKey(RouterKey s)	The following function is used to generate a VeniceMultiGetPath or a VeniceComputePath for a given key, and the generated VenicePath will be used to compose router request and forward to storage node; This function will be triggered by DDS router framework.
VenicePath	substitutePartitionKey(Collection<RouterKey> s)	The following function is used to generate a VeniceMultiGetPath or a VeniceComputePath for a given key subset, and the generated VenicePath will be used to compose router request and forward to storage node; This function will be triggered by DDS router framework.

Methods inherited from class com.linkedin.venice.router.api.path.VenicePath

canRequestStorageNode, composeRouterRequest, getClientComputeHeader, getOriginalRequestStartTs, getPartitionKeys, getRequestSize, getRequestType, getResourceName, getResponseDecompressor, getStoreName, getStreamingRequestType, getTime, getVeniceApiVersionHeader, getVersionNumber, isLongTailRetryWithinBudget, isRetryRequest, isRetryRequestTooLate, isStreamingRequest, markStorageNodeAsFast, recordOriginalRequestStartTimestamp, recordRequest, requestStorageNode, setPartitionKeys, setRetryRequest, setRetryRequest, setupVeniceHeaders

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.alpini.router.api.ResourcePath

getLocation, getPartitionKey

Field Details

routerKeyMap

protected final Map<RouterKey,K> routerKeyMap

Constructor Details

VeniceMultiKeyPath

public VeniceMultiKeyPath(StoreVersionName storeVersionName,
 RouterRetryConfig retryConfig,
 RetryManager retryManager,
 VeniceResponseDecompressor responseDecompressor)

VeniceMultiKeyPath

public VeniceMultiKeyPath(StoreVersionName storeVersionName,
 Map<RouterKey,K> routerKeyMap,
 RouterRetryConfig retryConfig,
 RetryManager retryManager,
 VeniceResponseDecompressor responseDecompressor)

Method Details

initialize

public void initialize(String resourceName,
 List<ByteBuffer> keys,
 VenicePartitionFinder partitionFinder,
 int maxKeyCount,
 AggRouterHttpRequestStats stats)
                throws RouterException

Fill the router key map and the index2routerKey map.

Parameters:

resourceName -

keys - Multiple keys from client request; keys have been deserialized to ByteBuffer

partitionFinder -

maxKeyCount -

Throws:

RouterException

substitutePartitionKey

public VenicePath substitutePartitionKey(RouterKey s)

The following function is used to generate a VeniceMultiGetPath or a VeniceComputePath for a given key, and the generated VenicePath will be used to compose router request and forward to storage node; This function will be triggered by DDS router framework.

Specified by:

substitutePartitionKey in class VenicePath

Parameters:

s -

Returns:

substitutePartitionKey

public VenicePath substitutePartitionKey(@Nonnull
 Collection<RouterKey> s)

The following function is used to generate a VeniceMultiGetPath or a VeniceComputePath for a given key subset, and the generated VenicePath will be used to compose router request and forward to storage node; This function will be triggered by DDS router framework.

Specified by:

substitutePartitionKey in class VenicePath

Parameters:

s -

Returns:

composeRouterRequestInternal

public org.apache.http.client.methods.HttpUriRequest composeRouterRequestInternal(String storageNodeUri)

Specified by:

composeRouterRequestInternal in class VenicePath

getHttpMethod

public io.netty.handler.codec.http.HttpMethod getHttpMethod()

Specified by:

getHttpMethod in class VenicePath

getBody

public byte[] getBody()

Specified by:

getBody in class VenicePath

getLongTailRetryMaxRouteForMultiKeyReq

public int getLongTailRetryMaxRouteForMultiKeyReq()

isLongTailRetryAllowedForNewRequest

public boolean isLongTailRetryAllowedForNewRequest()

Overrides:

isLongTailRetryAllowedForNewRequest in class VenicePath

setupRetryRelatedInfo

protected void setupRetryRelatedInfo(VenicePath originalPath)

Overrides:

setupRetryRelatedInfo in class VenicePath

isSmartLongTailRetryEnabled

public boolean isSmartLongTailRetryEnabled()

Overrides:

isSmartLongTailRetryEnabled in class VenicePath

getSmartLongTailRetryAbortThresholdMs

public int getSmartLongTailRetryAbortThresholdMs()

Overrides:

getSmartLongTailRetryAbortThresholdMs in class VenicePath

getLongTailRetryThresholdMs

public int getLongTailRetryThresholdMs()

Specified by:

getLongTailRetryThresholdMs in class VenicePath

getRequestId

public long getRequestId()

Overrides:

getRequestId in class VenicePath

getHelixGroupId

public int getHelixGroupId()

Overrides:

getHelixGroupId in class VenicePath

setHelixGroupId

public void setHelixGroupId(int helixGroupId)

Overrides:

setHelixGroupId in class VenicePath

setChunkedWriteHandler

public void setChunkedWriteHandler(io.netty.channel.ChannelHandlerContext ctx,
 VeniceChunkedWriteHandler chunkedWriteHandler,
 RouterStats<AggRouterHttpRequestStats> routerStats)

Overrides:

setChunkedWriteHandler in class VenicePath

getChunkedResponse

public VeniceChunkedResponse getChunkedResponse()

Overrides:

getChunkedResponse in class VenicePath

createRouterRequestKey

protected abstract K createRouterRequestKey(ByteBuffer key,
 int keyIdx,
 int partitionId)

Create a router request key.

Parameters:

key -

keyIdx -

partitionId -

Returns:

An instance of ComputeRouterRequestKeyV1 for compute request; return an instance of MultiGetRouterRequestKeyV1 for multi-get

fixRetryRequestForSubPath

protected abstract VenicePath fixRetryRequestForSubPath(Map<RouterKey,K> routerKeyMap)

Parameters:

routerKeyMap -

Returns:

a sub-path with a new set of keys

serializeRouterRequest

protected abstract byte[] serializeRouterRequest()

For multi-get requests, simply serialize the set of RouterKey to bytes; for read compute requests, concatenate the compute request and the serialized keys

Returns: