Fix Maintenance Task Sorting and Refactor Log Persistence (#8199)

* fix float stepping

* do not auto refresh

* only logs when non 200 status

* fix maintenance task sorting and cleanup redundant handler logic

* Refactor log retrieval to persist to disk and fix slowness

- Move log retrieval to disk-based persistence in GetMaintenanceTaskDetail
- Implement background log fetching on task completion in worker_grpc_server.go
- Implement async background refresh for in-progress tasks
- Completely remove blocking gRPC calls from the UI path to fix 10s timeouts
- Cleanup debug logs and performance profiling code

* Ensure consistent deterministic sorting in config_persistence cleanup

* Replace magic numbers with constants and remove debug logs

- Added descriptive constants for truncation limits and timeouts in admin_server.go and worker_grpc_server.go
- Replaced magic numbers with these constants throughout the codebase
- Verified removal of stdout debug printing
- Ensured consistent truncation logic during log persistence

* Address code review feedback on history truncation and logging logic

- Fix AssignmentHistory double-serialization by copying task in GetMaintenanceTaskDetail
- Fix handleTaskCompletion logging logic (mutually exclusive success/failure logs)
- Remove unused Timeout field from LogRequestContext and sync select timeouts with constants
- Ensure AssignmentHistory is only provided in the top-level field for better JSON structure

* Implement goroutine leak protection and request deduplication

- Add request deduplication in RequestTaskLogs to prevent multiple concurrent fetches for the same task
- Implement safe cleanup in timeout handlers to avoid race conditions in pendingLogRequests map
- Add a 10s cooldown for background log refreshes in GetMaintenanceTaskDetail to prevent spamming
- Ensure all persistent log-fetching goroutines are bounded and efficiently managed

* Fix potential nil pointer panics in maintenance handlers

- Add nil checks for adminServer in ShowTaskDetail, ShowMaintenanceWorkers, and UpdateTaskConfig
- Update getMaintenanceQueueData to return a descriptive error instead of nil when adminServer is uninitialized
- Ensure internal helper methods consistently check for adminServer initialization before use

* Strictly enforce disk-only log reading

- Remove background log fetching from GetMaintenanceTaskDetail to prevent timeouts and network calls during page view
- Remove unused lastLogFetch tracking fields to clean up dead code
- Ensure logs are only updated upon task completion via handleTaskCompletion

* Refactor GetWorkerLogs to read from disk

- Update /api/maintenance/workers/:id/logs endpoint to use configPersistence.LoadTaskExecutionLogs
- Remove synchronous gRPC call RequestTaskLogs to prevent timeouts and bad gateway errors
- Ensure consistent log retrieval behavior across the application (disk-only)

* Fix timestamp parsing in log viewer

- Update task_detail.templ JS to handle both ISO 8601 strings and Unix timestamps
- Fix "Invalid time value" error when displaying logs fetched from disk
- Regenerate templates

* master: fallback to HDD if SSD volumes are full in Assign

* worker: improve EC detection logging and fix skip counters

* worker: add Sync method to TaskLogger interface

* worker: implement Sync and ensure logs are flushed before task completion

* admin: improve task log retrieval with retries and better timeouts

* admin: robust timestamp parsing in task detail view
This commit is contained in:
Chris Lu
2026-02-04 08:48:55 -08:00
committed by GitHub
parent 2ff1cd9fc9
commit 72a8f598f2
51 changed files with 499 additions and 241 deletions

View File

@@ -942,10 +942,48 @@ templ TaskDetail(data *maintenance.TaskDetailData) {
return;
}
// Format and display logs with structured fields
let logText = '';
// Helper function to format timestamps robustly
function formatTimestamp(timestamp) {
if (!timestamp) {
return 'N/A';
}
let date;
// Check if timestamp is a numeric string (e.g., "1738652668")
if (typeof timestamp === 'string' && /^\d+$/.test(timestamp)) {
const numericTimestamp = parseInt(timestamp, 10);
// Treat values > 10^10 as milliseconds, otherwise as seconds
date = numericTimestamp > 10000000000
? new Date(numericTimestamp)
: new Date(numericTimestamp * 1000);
} else if (typeof timestamp === 'string') {
// ISO date string
date = new Date(timestamp);
} else if (typeof timestamp === 'number') {
// Numeric timestamp (seconds or milliseconds)
date = timestamp > 10000000000
? new Date(timestamp)
: new Date(timestamp * 1000);
} else {
return 'N/A';
}
// Validate the date
if (isNaN(date.getTime())) {
return 'N/A';
}
return date.toISOString();
}
logs.forEach(entry => {
const timestamp = entry.timestamp ? new Date(entry.timestamp * 1000).toISOString() : 'N/A';
const timestamp = formatTimestamp(entry.timestamp);
const level = entry.level || 'INFO';
const message = entry.message || '';
@@ -1011,7 +1049,12 @@ templ TaskDetail(data *maintenance.TaskDetailData) {
let logContent = '';
if (data.logs && data.logs.length > 0) {
data.logs.forEach(entry => {
const timestamp = entry.timestamp ? new Date(entry.timestamp * 1000).toISOString() : 'N/A';
let timestamp;
if (typeof entry.timestamp === 'string') {
timestamp = new Date(entry.timestamp).toISOString();
} else {
timestamp = entry.timestamp ? new Date(entry.timestamp * 1000).toISOString() : 'N/A';
}
const level = entry.level || 'INFO';
const message = entry.message || '';