avoid repeated reading disk (#7369)

* avoid repeated reading disk

* checks both flush time AND read position advancement

* wait on cond

* fix reading

Gap detection and skipping to earliest memory time
Time-based reads that include events at boundary times for first reads (offset ≤ 0)
Aggregated subscriber wake-up via ListenersWaits signaling

* address comments
This commit is contained in:
Chris Lu
2025-10-24 01:26:42 -07:00
committed by GitHub
parent d220875ef4
commit 7d147f238c
6 changed files with 236 additions and 73 deletions

View File

@@ -270,7 +270,7 @@ func (logBuffer *LogBuffer) LoopProcessLogDataWithOffset(readerName string, star
return
}
// CRITICAL FIX: If we're reading offset-based and there's no data in LogBuffer,
// If we're reading offset-based and there's no data in LogBuffer,
// return ResumeFromDiskError to let Subscribe try reading from disk again.
// This prevents infinite blocking when all data is on disk (e.g., after restart).
if startPosition.IsOffsetBased {
@@ -355,7 +355,6 @@ func (logBuffer *LogBuffer) LoopProcessLogDataWithOffset(readerName string, star
continue
}
glog.V(4).Infof("Unmarshaled log entry %d: TsNs=%d, Offset=%d, Key=%s", batchSize+1, logEntry.TsNs, logEntry.Offset, string(logEntry.Key))
// Handle offset-based filtering for offset-based start positions
@@ -377,7 +376,7 @@ func (logBuffer *LogBuffer) LoopProcessLogDataWithOffset(readerName string, star
// println("stopTsNs", stopTsNs, "logEntry.TsNs", logEntry.TsNs)
return
}
// CRITICAL FIX: Use logEntry.Offset + 1 to move PAST the current entry
// Use logEntry.Offset + 1 to move PAST the current entry
// This prevents infinite loops where we keep requesting the same offset
lastReadPosition = NewMessagePosition(logEntry.TsNs, logEntry.Offset+1)