Class NettyP2PBlobTransferManager
- java.lang.Object
-
- com.linkedin.davinci.blobtransfer.NettyP2PBlobTransferManager
-
- All Implemented Interfaces:
BlobTransferManager<java.lang.Void>
,P2PBlobTransferManager<java.lang.Void>
,java.lang.AutoCloseable
public class NettyP2PBlobTransferManager extends java.lang.Object implements P2PBlobTransferManager<java.lang.Void>
Netty based P2P blob transfer manager implementation. Upon start, it starts the blob transfer service and the client, so it can receive requests from peers to serve blobs and in the meanwhile, it can make requests to other peers to fetch blobs.
-
-
Field Summary
Fields Modifier and Type Field Description protected static int
MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST
protected static int
MAX_TIMEOUT_FOR_BLOB_TRANSFER_IN_MIN
protected NettyFileTransferClient
nettyClient
protected BlobFinder
peerFinder
-
Constructor Summary
Constructors Constructor Description NettyP2PBlobTransferManager(P2PBlobTransferService blobTransferService, NettyFileTransferClient nettyClient, BlobFinder peerFinder)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
Close the blob transfer manager and related resourcesjava.util.concurrent.CompletionStage<java.io.InputStream>
get(java.lang.String storeName, int version, int partition)
Get the blobs for the given storeName and partition error cases: 1.void
start()
Start the blob transfer manager and related resources-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface com.linkedin.davinci.blobtransfer.P2PBlobTransferManager
put
-
-
-
-
Field Detail
-
MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST
protected static final int MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST
- See Also:
- Constant Field Values
-
MAX_TIMEOUT_FOR_BLOB_TRANSFER_IN_MIN
protected static final int MAX_TIMEOUT_FOR_BLOB_TRANSFER_IN_MIN
- See Also:
- Constant Field Values
-
nettyClient
protected final NettyFileTransferClient nettyClient
-
peerFinder
protected final BlobFinder peerFinder
-
-
Constructor Detail
-
NettyP2PBlobTransferManager
public NettyP2PBlobTransferManager(P2PBlobTransferService blobTransferService, NettyFileTransferClient nettyClient, BlobFinder peerFinder)
-
-
Method Detail
-
start
public void start() throws java.lang.Exception
Description copied from interface:BlobTransferManager
Start the blob transfer manager and related resources- Specified by:
start
in interfaceBlobTransferManager<java.lang.Void>
- Throws:
java.lang.Exception
-
get
public java.util.concurrent.CompletionStage<java.io.InputStream> get(java.lang.String storeName, int version, int partition) throws VenicePeersNotFoundException
Get the blobs for the given storeName and partition error cases: 1. [Fatal Case] If no peers info are found for the requested blob, a VenicePeersNotFoundException is thrown. In this case, blob transfer is not used for bootstrapping at all. 2. If one host connect error, it will throw VenicePeersCannotConnectException and retry connecting to the peer again After MAX_RETRIES_FOR_BLOB_TRANSFER_PER_HOST times, if still failed to connect, move to the next possible host. 3. If the connected host does not have the requested file, a VeniceBlobTransferFileNotFoundException is thrown, and the process moves on to the next available host. 4. [Fatal Case] If any unexpected exception occurs, such as InterruptedException, ExecutionException, or TimeoutException during the file/metadata transfer, a VeniceException is thrown, and blob transfer is skipped for bootstrapping to save time. 5. [Fatal Case] If all peers fail to connect or have no snapshot, a VenicePeersNotFoundException is thrown, and Kafka is used for bootstrapping instead. success case: 1. If the blob is successfully fetched from a peer, an InputStream of the blob is returned.- Specified by:
get
in interfaceBlobTransferManager<java.lang.Void>
- Parameters:
storeName
- the name of the storeversion
- the version of the storepartition
- the partition of the store- Returns:
- the InputStream of the blob
- Throws:
VenicePeersNotFoundException
-
close
public void close() throws java.lang.Exception
Description copied from interface:BlobTransferManager
Close the blob transfer manager and related resources- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfaceBlobTransferManager<java.lang.Void>
- Throws:
java.lang.Exception
-
-