Package com.linkedin.davinci.blobtransfer

Class NettyP2PBlobTransferManager

java.lang.Object

com.linkedin.davinci.blobtransfer.NettyP2PBlobTransferManager

All Implemented Interfaces:

BlobTransferManager<java.lang.Void>, P2PBlobTransferManager<java.lang.Void>, java.lang.AutoCloseable

public class NettyP2PBlobTransferManager
extends java.lang.Object
implements P2PBlobTransferManager<java.lang.Void>

Netty based P2P blob transfer manager implementation. Upon start, it starts the blob transfer service and the client, so it can receive requests from peers to serve blobs and in the meanwhile, it can make requests to other peers to fetch blobs.

Field Summary

Fields

Modifier and Type	Field	Description
protected static int	MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST
protected static int	MAX_TIMEOUT_FOR_BLOB_TRANSFER_IN_MIN
protected NettyFileTransferClient	nettyClient
protected BlobFinder	peerFinder

Constructor Summary

Constructors

Constructor	Description
NettyP2PBlobTransferManager(P2PBlobTransferService blobTransferService, NettyFileTransferClient nettyClient, BlobFinder peerFinder)

Method Summary

Modifier and Type	Method	Description
void	close()	Close the blob transfer manager and related resources
java.util.concurrent.CompletionStage<java.io.InputStream>	get(java.lang.String storeName, int version, int partition)	Get the blobs for the given storeName and partition error cases: 1.
void	start()	Start the blob transfer manager and related resources

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.linkedin.davinci.blobtransfer.P2PBlobTransferManager

put

Field Detail

MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST

protected static final int MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST

See Also:

Constant Field Values

MAX_TIMEOUT_FOR_BLOB_TRANSFER_IN_MIN

protected static final int MAX_TIMEOUT_FOR_BLOB_TRANSFER_IN_MIN

See Also:

Constant Field Values

nettyClient

protected final NettyFileTransferClient nettyClient

peerFinder

protected final BlobFinder peerFinder

Constructor Detail

NettyP2PBlobTransferManager

public NettyP2PBlobTransferManager(P2PBlobTransferService blobTransferService,
                                   NettyFileTransferClient nettyClient,
                                   BlobFinder peerFinder)

Method Detail

start

public void start()
           throws java.lang.Exception

Description copied from interface: BlobTransferManager

Start the blob transfer manager and related resources

Specified by:

start in interface BlobTransferManager<java.lang.Void>

Throws:

java.lang.Exception

get

public java.util.concurrent.CompletionStage<java.io.InputStream> get(java.lang.String storeName,
                                                                     int version,
                                                                     int partition)
                                                              throws VenicePeersNotFoundException

Get the blobs for the given storeName and partition error cases: 1. [Fatal Case] If no peers info are found for the requested blob, a VenicePeersNotFoundException is thrown. In this case, blob transfer is not used for bootstrapping at all. 2. If one host connect error, it will throw VenicePeersCannotConnectException and retry connecting to the peer again After MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST times, if still failed to connect, move to the next possible host. 3. If the connected host does not have the requested file, a VeniceBlobTransferFileNotFoundException is thrown, and the process moves on to the next available host. 4. [Fatal Case] If any unexpected exception occurs, such as InterruptedException, ExecutionException, or TimeoutException during the file/metadata transfer, a VeniceException is thrown, and blob transfer is skipped for bootstrapping to save time. 5. [Fatal Case] If all peers fail to connect or have no snapshot, a VenicePeersNotFoundException is thrown, and Kafka is used for bootstrapping instead. success case: 1. If the blob is successfully fetched from a peer, an InputStream of the blob is returned.

Specified by:

get in interface BlobTransferManager<java.lang.Void>

Parameters:

storeName - the name of the store

version - the version of the store

partition - the partition of the store

Returns:

the InputStream of the blob

Throws:

VenicePeersNotFoundException

close

public void close()
           throws java.lang.Exception

Description copied from interface: BlobTransferManager

Close the blob transfer manager and related resources

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface BlobTransferManager<java.lang.Void>

Throws:

java.lang.Exception