Try Documentalist,
my app that offers fast, offline access to 190+ programmer API docs.
What is dailyrotate?
dailyrotate is a Go library for rotating files daily.
References:
- code: github.com/kjk/dailyrotate
- api docs: https://godoc.org/github.com/kjk/dailyrotate
- examples: https://github.com/kjk/dailyrotate/tree/master/examples
Imagine you're writing a backend for a web app in Go.
Logging is important for observability and debugability.
Web backends are long-running and you don't want your logs to grow without bounds.
Log rotation solves the issue of infinite growth: when the log file grows beyond a certain size or at certain time, a new log file is created.
To implement log rotation you can use external programs like
logrotate but I prefer simpler solution and thanks to io.Writer interface, Go makes implementing log rotation simple.
dailyrotate is a Go library that makes it easy to implement log rotation.
By defaultd logs rotate daily, at midnight UTC time.
I use
dailyrotate in production (for example in presstige.io).
How to use dailyrotate
You can see full code of the example.
Open log file
Logging happens everywhere in the code so typically we would have a global variable for the log file and open the log at program start
var (
logFile *dailyrotate.File
)
func openLogFile(pathFormat string, onClose func(string, bool)) error {
w, err := dailyrotate.NewFile(pathFormat, onLogClose)
if err != nil {
return err
}
logFile = w
return nil
}
func main() {
logDir := "logs"
// we have to ensure the directory we want to write to
// already exists
err := os.MkdirAll(logDir, 0755)
if err != nil {
log.Fatalf("os.MkdirAll()(")
}
pathFormat := filepath.Join(logDir, "2006-01-02.txt")
err = openLogFile(pathFormat, onLogClose)
if err != nil {
log.Fatalf("openLogFile failed with '%s'\n", err)
}
// ... the rest of your program
}Just like for regular
os.Create() we have to ensure that the directory for log files exists with os.MkdirAll(dir, 0755).
Rotating files implies that the name of the file changes.
I like the convention of using date in the name of the file so
dailyrotate uses time.Format formatting layout for file paths.
I use
2006-01-02.txt for the file format (which is YYYY-MM-DD + .txt).
It's easy to locate a log for a given day and the names sort by day.
Log file is opened in append mode.
Write to log file
dailyrotate.File implements io.Writer so to write we uses the standard f.Write(d []byte) (int, error).
func writeToLog(msg string) error {
_, err := logFile.Write([]byte(msg))
return err
}Close log file
dailyrotate.File implements io.Close so to close we use Close() error.
It's safe to call
Close multiple times.
func closeLogFile() error {
return logFile.Close()
}Execute code on log rotation
Imagine that when a log rotates you want to immediately backup the file to online storage like S3.
dailyrotate.NewFile takes an optional function that will be called when the log file is closed. A skeleton of a callback:
func onLogClose(path string, didRotate bool) {
fmt.Printf("we just closed a file '%s', didRotate: %v\n", path, didRotate)
if !didRotate {
return
}
// process just closed file e.g. upload to S3 for backup
go func() {
// if processing takes a long time, do it in background
}()
}The file can be closed either because you called
Close explicitly (e.g. because the program is exiting) or implicitly due to rotation.
You can distinguish the 2 cases with
didRotate argument.
Rotate at a different time
By default logs rotate at midnight UTC time. We use UTC because "midnight" means different time in different timezones. In California, it's 5 PM and in Paris it's 2 AM.
You can change UTC to a different time zone by setting
File.Location to a timezone Location (which you can load with time.LoadLocation):
loc, errr := time.LoadLocation("America/Los_Angeles")
if err != nil {
log.Fatalf("time.LoadLocation() failed with '%s'\n", err)
}
f, err := dailyrotate.NewFile("2006-01-02.txt", nil)
if err != nil {
log.Fatalf("dailyrotate.NewFile() failed with '%s'\n", err)
}
f.Location = loc