Package com.linkedin.venice.router.streaming

Class NettyChunkedWriteHandler

java.lang.Object
io.netty.channel.ChannelHandlerAdapter
io.netty.channel.ChannelInboundHandlerAdapter
io.netty.channel.ChannelDuplexHandler
com.linkedin.venice.router.streaming.NettyChunkedWriteHandler
All Implemented Interfaces:
io.netty.channel.ChannelHandler, io.netty.channel.ChannelInboundHandler, io.netty.channel.ChannelOutboundHandler
Direct Known Subclasses:
VeniceChunkedWriteHandler
public class NettyChunkedWriteHandler extends io.netty.channel.ChannelDuplexHandler
This class was copied from ChunkedWriteHandler, and the only change we made here is to using ConcurrentLinkedQueue for queue since resumeTransfer() or discard(Throwable) could happen in any sequence with any parallelism because of scatter/gather logic in Venice Router. TODO: sync up with Netty team to see whether we could incorporate this change into Netty code base or not. Original Javadoc starts from here: A ChannelHandler that adds support for writing a large data stream asynchronously neither spending a lot of memory nor getting OutOfMemoryError. Large data streaming such as file transfer requires complicated state management in a ChannelHandler implementation. ChunkedWriteHandler manages such complicated states so that you can send a large data stream without difficulties.

To use ChunkedWriteHandler in your application, you have to insert a new ChunkedWriteHandler instance: 

 ChannelPipeline p = ...;
 p.addLast("streamer", new ChunkedWriteHandler());
 p.addLast("handler", new MyHandler());
Once inserted, you can write a ChunkedInput so that the ChunkedWriteHandler can pick it up and fetch the content of the stream chunk by chunk and write the fetched chunk downstream: 
 Channel ch = ...;
 ch.write(new ChunkedFile(new File("video.mkv"));

Sending a stream which generates a chunk intermittently

Some ChunkedInput generates a chunk on a certain event or timing. Such ChunkedInput implementation often returns null on ChunkedInput.readChunk(ChannelHandlerContext), resulting in the indefinitely suspended transfer. To resume the transfer when a new chunk is available, you have to call resumeTransfer().

  • Nested Class Summary

    Nested classes/interfaces inherited from interface io.netty.channel.ChannelHandler

    io.netty.channel.ChannelHandler.Sharable

  • Constructor Summary

    Constructors
    Constructor
    Description
    NettyChunkedWriteHandler()
     
    NettyChunkedWriteHandler(int maxPendingWrites)
    Deprecated.
    use NettyChunkedWriteHandler()

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    channelInactive(io.netty.channel.ChannelHandlerContext ctx)
     
    void
    channelWritabilityChanged(io.netty.channel.ChannelHandlerContext ctx)
     
    void
    flush(io.netty.channel.ChannelHandlerContext ctx)
     
    void
    handlerAdded(io.netty.channel.ChannelHandlerContext ctx)
     
    void
    resumeTransfer()
    Continues to fetch the chunks from the input.
    void
    write(io.netty.channel.ChannelHandlerContext ctx, Object msg, io.netty.channel.ChannelPromise promise)
     

    Methods inherited from class io.netty.channel.ChannelDuplexHandler

    bind, close, connect, deregister, disconnect, read

    Methods inherited from class io.netty.channel.ChannelInboundHandlerAdapter

    channelActive, channelRead, channelReadComplete, channelRegistered, channelUnregistered, exceptionCaught, userEventTriggered

    Methods inherited from class io.netty.channel.ChannelHandlerAdapter

    ensureNotSharable, handlerRemoved, isSharable

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface io.netty.channel.ChannelHandler

    handlerRemoved

  • Constructor Details

    • NettyChunkedWriteHandler

      public NettyChunkedWriteHandler()

    • NettyChunkedWriteHandler

      @Deprecated public NettyChunkedWriteHandler(int maxPendingWrites)
      Deprecated.
      use NettyChunkedWriteHandler()

  • Method Details

    • handlerAdded

      public void handlerAdded(io.netty.channel.ChannelHandlerContext ctx) throws Exception
      Specified by:
      handlerAdded in interface io.netty.channel.ChannelHandler
      Overrides:
      handlerAdded in class io.netty.channel.ChannelHandlerAdapter
      Throws:
      Exception

    • resumeTransfer

      public void resumeTransfer()
      Continues to fetch the chunks from the input.

    • write

      public void write(io.netty.channel.ChannelHandlerContext ctx, Object msg, io.netty.channel.ChannelPromise promise) throws Exception
      Specified by:
      write in interface io.netty.channel.ChannelOutboundHandler
      Overrides:
      write in class io.netty.channel.ChannelDuplexHandler
      Throws:
      Exception

    • flush

      public void flush(io.netty.channel.ChannelHandlerContext ctx) throws Exception
      Specified by:
      flush in interface io.netty.channel.ChannelOutboundHandler
      Overrides:
      flush in class io.netty.channel.ChannelDuplexHandler
      Throws:
      Exception

    • channelInactive

      public void channelInactive(io.netty.channel.ChannelHandlerContext ctx) throws Exception
      Specified by:
      channelInactive in interface io.netty.channel.ChannelInboundHandler
      Overrides:
      channelInactive in class io.netty.channel.ChannelInboundHandlerAdapter
      Throws:
      Exception

    • channelWritabilityChanged

      public void channelWritabilityChanged(io.netty.channel.ChannelHandlerContext ctx) throws Exception
      Specified by:
      channelWritabilityChanged in interface io.netty.channel.ChannelInboundHandler
      Overrides:
      channelWritabilityChanged in class io.netty.channel.ChannelInboundHandlerAdapter
      Throws:
      Exception