Writing to a file robustly is tricky:
- we need to check errors from
Write()
- we need to
Sync() before Close(), and check the error
- we need to check error from
Close()
- if any of the above returned an error, the file is corrupted so we should remove it
To open a file for writing use atomicfile.New() instead of os.Create() or os.OpenFile().
It returns an atomicfile.File struct which implements io.Reader, io.Closer and io.ReadCloser interfaces so it can be used by most code that accepted os.File.
func writeToFileAtomically(filePath string, data []byte) error {
f, err := atomicfile.New(filePath)
if err != nil {
return err
}
// ensure that if there's a panic or early exit due to
// error, the file will not be created
// RemoveIfNotClosed() after Close() is a no-op
defer f.RemoveIfNotClosed()
_, err = f.Write(data)
if err != nil {
return err
}
return f.Close()
}
For proper cleanup, we do defer f.RemoveIfNotClose().
It’ll ensure that temporary file is removed if we never call Close() (e.g. due to a panic or early exit due to error).
Why it’s tricky
Here’s file writing code that has a subtle bug:
func writeToFileAtomically(filePath string, data []byte) error {
f, err := os.Create(filePath)
if err != nil {
return err
}
defer f.Close()
_, err = f.Write(data)
if err != nil {
return err
}
return nil
}
There are problems with this code:
- if
Write() fails, we create a corrupted (partially written) file. We should os.Remove() it in case of errors
Close() can return an error. Writes to files are buffered so Close() might need to write buffered data which can fail. Forgetting to handle an error from Close() is a common mistake when writing to files- even if
Close() returns no error, the filesystem cache might not be written to disk, so we need to call Sync() before Close()
- it anything fails, we should delete the partially written file
- ideally we shouldn’t expose partially written file until it’s fully written
How atomicfile works
To avoid exposing partially written file, atomicfile writes to a temporary file.
Only after we finish writing, call Close() and there were no errors, we rename temporary file to final name.
The limitation of this approach is that it only works for new files. It doesn’t support appending.
Calling Close() multiple times is ok. You should check the error returned by Close() but you can also use defer f.Close() to ensure it’s closed on error conditions and panics.
References
Those issues are surprisingly tricky. Some people investigated them in depth: