Package com.linkedin.davinci.stats

Class AggVersionedIngestionStats

java.lang.Object
com.linkedin.davinci.stats.AbstractVeniceAggVersionedStats<IngestionStats,IngestionStatsReporter>
com.linkedin.davinci.stats.AggVersionedIngestionStats
All Implemented Interfaces:
StoreDataChangedListener
public class AggVersionedIngestionStats extends AbstractVeniceAggVersionedStats<IngestionStats,IngestionStatsReporter>
The store level stats or the total stats will be unpopulated because there is no easy and reliable way to aggregate gauge stats such as rt topic offset lag.

OTel stats lifecycle: OTel stats are created lazily by getIngestionOtelStats(java.lang.String) and updated by onVersionInfoUpdated(java.lang.String, int, int) via computeIfPresent. This class uses eager loading (loadAllStats() is NOT overridden), so onVersionInfoUpdated and cleanupVersionResources need null guards because they are called during the super() constructor before subclass fields (otelStatsMap) are initialized.

  • Constructor Details

  • Method Details

    • onVersionInfoUpdated

      protected void onVersionInfoUpdated(String storeName, int currentVersion, int futureVersion)
      Updates version info for existing OTel stats only. Null guard needed because eager loading calls this from the super() constructor before otelStatsMap is initialized.
      Overrides:
      onVersionInfoUpdated in class AbstractVeniceAggVersionedStats<IngestionStats,IngestionStatsReporter>

    • cleanupVersionResources

      protected void cleanupVersionResources(String storeName, int version)
      Description copied from class: AbstractVeniceAggVersionedStats
      Hook method for subclasses to clean up version-specific resources (e.g., OTel stats) when a version is removed. Same re-entrance warning as AbstractVeniceAggVersionedStats.onVersionInfoUpdated(java.lang.String, int, int).
      Overrides:
      cleanupVersionResources in class AbstractVeniceAggVersionedStats<IngestionStats,IngestionStatsReporter>

    • handleStoreDeleted

      public void handleStoreDeleted(String storeName)
      Specified by:
      handleStoreDeleted in interface StoreDataChangedListener
      Overrides:
      handleStoreDeleted in class AbstractVeniceAggVersionedStats<IngestionStats,IngestionStatsReporter>

    • setIngestionTask

      public void setIngestionTask(String storeVersionTopic, StoreIngestionTask ingestionTask)

    • recordRecordsConsumed

      public void recordRecordsConsumed(String storeName, int version)
      Records generic records consumed metric (Tehuti only).

      OTel metrics are intentionally NOT recorded here to avoid double-counting. OTel consumption metrics include a ReplicaType dimension (LEADER/FOLLOWER) and are recorded by recordLeaderConsumed(java.lang.String, int, long) and recordFollowerConsumed(java.lang.String, int, long) instead.

    • recordBytesConsumed

      public void recordBytesConsumed(String storeName, int version, long bytes)
      Records generic bytes consumed metric (Tehuti only).

      OTel metrics are intentionally NOT recorded here to avoid double-counting. OTel consumption metrics include a ReplicaType dimension (LEADER/FOLLOWER) and are recorded by recordLeaderConsumed(java.lang.String, int, long) and recordFollowerConsumed(java.lang.String, int, long) instead.

    • recordLeaderConsumed

      public void recordLeaderConsumed(String storeName, int version, long bytes)

    • recordFollowerConsumed

      public void recordFollowerConsumed(String storeName, int version, long bytes)

    • recordLeaderProduced

      public void recordLeaderProduced(String storeName, int version, long bytesProduced, int recordCount)

    • recordRegionHybridConsumption

      public void recordRegionHybridConsumption(String storeName, int version, int regionId, long bytesConsumed, long currentTimeMs, String sourceRegion, VeniceRegionLocality regionLocality)

    • recordUpdateIgnoredDCR

      public void recordUpdateIgnoredDCR(String storeName, int version)

    • recordTotalDCR

      public void recordTotalDCR(String storeName, int version)

    • recordTotalDuplicateKeyUpdate

      public void recordTotalDuplicateKeyUpdate(String storeName, int version)

    • recordTimestampRegressionDCRError

      public void recordTimestampRegressionDCRError(String storeName, int version)

    • recordOffsetRegressionDCRError

      public void recordOffsetRegressionDCRError(String storeName, int version)

    • recordTombStoneCreationDCR

      public void recordTombStoneCreationDCR(String storeName, int version)

    • setIngestionTaskPushTimeoutGauge

      public void setIngestionTaskPushTimeoutGauge(String storeName, int version)

    • resetIngestionTaskPushTimeoutGauge

      public void resetIngestionTaskPushTimeoutGauge(String storeName, int version)

    • recordSubscribePrepLatency

      public void recordSubscribePrepLatency(String storeName, int version, double value)

    • recordProducerCallBackLatency

      public void recordProducerCallBackLatency(String storeName, int version, double value, long currentTimeMs)

    • recordLeaderPreprocessingLatency

      public void recordLeaderPreprocessingLatency(String storeName, int version, double value, long currentTimeMs)

    • recordInternalPreprocessingLatency

      public void recordInternalPreprocessingLatency(String storeName, int version, double value, long currentTimeMs)

    • recordLeaderLatencies

      public void recordLeaderLatencies(String storeName, int version, long currentTimeMs, double producerBrokerLatencyMs, double brokerConsumerLatencyMs)

    • recordFollowerLatencies

      public void recordFollowerLatencies(String storeName, int version, long currentTimeMs, double producerBrokerLatencyMs, double brokerConsumerLatencyMs)

    • recordLeaderProducerCompletionTime

      public void recordLeaderProducerCompletionTime(String storeName, int version, double value, long currentTimeMs)

    • recordConsumedRecordEndToEndProcessingLatency

      public void recordConsumedRecordEndToEndProcessingLatency(String storeName, int version, double value, long currentTimeMs)

    • recordNearlineProducerToLocalBrokerLatency

      public void recordNearlineProducerToLocalBrokerLatency(String storeName, int version, double value, long timestamp)

    • recordMaxIdleTime

      public void recordMaxIdleTime(String storeName, int version, long idleTimeMs, boolean emitTehuti)

    • recordBatchProcessingRequest

      public void recordBatchProcessingRequest(String storeName, int version, int size, long timestamp)

    • recordBatchProcessingRequestError

      public void recordBatchProcessingRequestError(String storeName, int version)

    • recordBatchProcessingLatency

      public void recordBatchProcessingLatency(String storeName, int version, double latency, long timestamp)

    • recordConsumerQueuePutTime

      public void recordConsumerQueuePutTime(String storeName, int version, double latencyMs)

    • recordStorageEnginePutTime

      public void recordStorageEnginePutTime(String storeName, int version, double latencyMs)

    • recordStorageEngineDeleteTime

      public void recordStorageEngineDeleteTime(String storeName, int version, double latencyMs)

    • recordConsumerActionTime

      public void recordConsumerActionTime(String storeName, int version, double latencyMs)

    • recordLongRunningTaskCheckTime

      public void recordLongRunningTaskCheckTime(String storeName, int version, double latencyMs)

    • recordViewWriterProduceTime

      public void recordViewWriterProduceTime(String storeName, int version, double latencyMs)

    • recordViewWriterAckTime

      public void recordViewWriterAckTime(String storeName, int version, double latencyMs)

    • recordProducerEnqueueTime

      public void recordProducerEnqueueTime(String storeName, int version, double latencyMs)

    • recordProducerCompressTime

      public void recordProducerCompressTime(String storeName, int version, double latencyMs)

    • recordProducerSynchronizeTime

      public void recordProducerSynchronizeTime(String storeName, int version, double latencyMs)

    • recordPartialUpdateTime

      public void recordPartialUpdateTime(String storeName, int version, VenicePartialUpdateOperation op, double latencyMs)

    • recordDcrLookupTime

      public void recordDcrLookupTime(String storeName, int version, VeniceRecordType recordType, double latencyMs)

    • recordDcrMergeTime

      public void recordDcrMergeTime(String storeName, int version, VeniceDCROperation op, double latencyMs)

    • recordUnexpectedMessageCount

      public void recordUnexpectedMessageCount(String storeName, int version)

    • recordStoreMetadataInconsistentCount

      public void recordStoreMetadataInconsistentCount(String storeName, int version)

    • recordResubscriptionFailureCount

      public void recordResubscriptionFailureCount(String storeName, int version)

    • recordPartialUpdateCacheHitCount

      public void recordPartialUpdateCacheHitCount(String storeName, int version)

    • recordChecksumVerificationFailureCount

      public void recordChecksumVerificationFailureCount(String storeName, int version)

    • recordIngestionFailureCount

      public void recordIngestionFailureCount(String storeName, int version, VeniceIngestionFailureReason reason)

    • recordDcrLookupCacheHitCount

      public void recordDcrLookupCacheHitCount(String storeName, int version, VeniceRecordType recordType)

    • recordBytesConsumedAsUncompressedSize

      public void recordBytesConsumedAsUncompressedSize(String storeName, int version, long bytes)

    • recordKeySize

      public void recordKeySize(String storeName, int version, long bytes)

    • recordValueSize

      public void recordValueSize(String storeName, int version, long bytes)

    • recordAssembledSize

      public void recordAssembledSize(String storeName, int version, VeniceRecordType recordType, long bytes)

    • recordAssembledSizeRatio

      public void recordAssembledSizeRatio(String storeName, int version, double ratio)