Package com.linkedin.venice.utils

Class DiskUsage

  • public class DiskUsage
extends java.lang.Object
    We define a threshold where we consider the disk to be full. For example, we might declare the disk full at 0.90 usage. Each task that writes to the disk maintains an instance of this class. That instance calculates a "reserve" of space that the task is allowed to write without checking the disk again. For example, a 3TB (or 3000GB) disk with a threshold of 0.90 requires 300G to be free on the disk. We calculate the reserve space as 0.001 of the required free space. In this case 300MB. On every write, the task calls this object's #isDiskFull() method, passing the number of bytes written. Once the task has written 300MB (or whatever the reserve is) then this object will reinspect the disk for it's actual usage and reallocate a reserve. If we have exceeded the disk full threshold, we report the disk is full. We calculate reserve space as 0.001 of required free space to allow 1000 concurrent tasks to write to disk without risk of blowing past the disk full threshold and actually reaching 100% disk usage.

    • Constructor Summary

      Constructors 
      Constructor Description
      DiskUsage​(java.lang.String path, double diskFullThreshold)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.lang.String getDiskStatus()  
      boolean isDiskFull​(long bytesWritten)
      Each time you write bytes to the disk, call this method.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • DiskUsage

        public DiskUsage​(java.lang.String path,
                 double diskFullThreshold)

    • Method Detail

      • isDiskFull

        public boolean isDiskFull​(long bytesWritten)
        Each time you write bytes to the disk, call this method. It will determine an appropriate frequency to update its view of the disk so this call will usually be very fast. If the disk is beyond the threshold that was set at construction then it will return true. Otherwise it returns false.

      • getDiskStatus

        public java.lang.String getDiskStatus()