Browse Source

Reduce task logger glog noise and remove per-write fsync (#8603)

* Reduce task logger noise: stop duplicating every log entry to glog and stderr

Every task log entry was being tripled: written to the task log file,
forwarded to glog (which writes to /tmp by default with no rotation),
and echoed to stderr. This caused glog files to fill /tmp on long-running
workers.

- Remove INFO/DEBUG forwarding to glog (only ERROR/WARNING remain)
- Remove stderr echo of every log line
- Remove fsync on every single log write (unnecessary for log files)

* Fix glog call depth for correct source file attribution

The call stack is: caller → Error() → log() → writeLogEntry() →
glog.ErrorDepth(), so depth=4 is needed for glog to report the
original caller's file and line number.
pull/8605/head
Chris Lu 2 days ago
committed by GitHub
parent
commit
2ff4a07544
No known key found for this signature in database GPG Key ID: B5690EEEBB952194
  1. 17
      weed/worker/tasks/task_logger.go

17
weed/worker/tasks/task_logger.go

@ -317,26 +317,15 @@ func (l *FileTaskLogger) writeLogEntry(entry TaskLogEntry) {
return
}
// Flush to disk
if err := l.logFile.Sync(); err != nil {
glog.Errorf("Failed to sync log file: %v", err)
}
// Also log to console and stderr if enabled
// Only forward errors and warnings to glog; routine task logs stay in the task log file.
if l.config.EnableConsole {
// Log to glog with proper call depth to show actual source location
// We need depth 3 to skip: writeLogEntry -> log -> Info/Warning/Error calls to reach the original caller
formattedMsg := fmt.Sprintf("[TASK-%s] %s: %s", l.taskID, entry.Level, entry.Message)
switch entry.Level {
case "ERROR":
glog.ErrorDepth(3, formattedMsg)
glog.ErrorDepth(4, formattedMsg)
case "WARNING":
glog.WarningDepth(3, formattedMsg)
default: // INFO, DEBUG, etc.
glog.InfoDepth(3, formattedMsg)
glog.WarningDepth(4, formattedMsg)
}
// Also log to stderr for immediate visibility
fmt.Fprintf(os.Stderr, "[TASK-%s] %s: %s\n", l.taskID, entry.Level, entry.Message)
}
}

Loading…
Cancel
Save