Package com.linkedin.davinci.blobtransfer

Class BlobTransferPooledByteBufAllocator

java.lang.Object

com.linkedin.davinci.blobtransfer.BlobTransferPooledByteBufAllocator

public final class BlobTransferPooledByteBufAllocator
extends Object

Supplies the PooledByteBufAllocator that blob transfer channels allocate their buffers from.

Netty channels allocate from the process-wide PooledByteBufAllocator.DEFAULT unless given their own, so blob transfer's usage is otherwise indistinguishable from every other Netty user in the process. A dedicated pool makes allocator.metric().usedDirectMemory() attributable to blob transfer alone. It covers only buffers that come from this allocator, so the metadata path's byte[] and Unpooled buffers are not counted.

Method Summary

Modifier and Type	Method	Description
static io.netty.buffer.PooledByteBufAllocator	create(String role, boolean dedicatedAllocatorEnabled, int eventLoopThreadCount)
static String	describeUsage(io.netty.buffer.ByteBufAllocator allocator)	Renders the memory charged to allocator at the moment of the call, or null when it is not a dedicated blob transfer allocator, since PooledByteBufAllocator.DEFAULT describes the whole process.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

create

public static io.netty.buffer.PooledByteBufAllocator create(String role,
 boolean dedicatedAllocatorEnabled,
 int eventLoopThreadCount)

Parameters:

role - which end of blob transfer owns the allocator, so the chunk size logged below is attributable.

dedicatedAllocatorEnabled - when false, returns PooledByteBufAllocator.DEFAULT, which is what a channel uses when no allocator is configured, so installing it changes nothing.

eventLoopThreadCount - the size of the event loop pool the allocator serves. Both arena counts are set to it so the pool scales with blob transfer's event loop pool rather than Netty's process-wide cores * 2. Netty hands a thread the least used arena rather than one arena each, so sizing the arenas to the threads spreads them without reserving arenas no thread will use.

describeUsage

public static String describeUsage(io.netty.buffer.ByteBufAllocator allocator)

Renders the memory charged to allocator at the moment of the call, or null when it is not a dedicated blob transfer allocator, since PooledByteBufAllocator.DEFAULT describes the whole process.

usedDirectMemory and usedHeapMemory count whole reserved chunks rather than the bytes in use, which is the figure that matters against -XX:MaxDirectMemorySize. A buffer larger than the chunk size is allocated outside the pool and leaves both fields as soon as it is released, so on a deployment that narrows the chunk through -Dio.netty.allocator.maxOrder a sample taken once a transfer has settled can read close to the baseline: these are instantaneous samples, not peaks.