Overview of the problem
Error handling has always been a touchy topic in Go. Rather than a try-catch form of error handling, Go prefers to go with explicit returning of errors.
Let me preface this article by saying that this is a great philosophy since it forces developers to handle errors manually rather than running into unhandled exceptions at runtime.
BUT… it comes with some major drawbacks. Look at this code:
func CopyFile(src, dst string) (int, error) {
r, err := os.Open(src)
if err != nil {
return 0, err
}
defer r.Close()
w, err := os.Create(dst)
if err != nil {
return 0, err
}
n, err := io.Copy(w, r)
if err != nil {
w.Close()
os.Remove(dst)
return 0, err
}
if err := w.Close(); err != nil {
os.Remove(dst)
return 0, err
}
return n, nil
}
That’s a lot of error checking for a relatively small piece of logic!
This particular example is taken from the official proposal for error handling in Go2, and its not the only proposal out there. The proposal also hands out a list of issue on this: #21161 , #18721 , #16225 , #21182 , #19727 , #19642 , #19991 .
Clearly its a problem many developers have banged their heads against.
How Google solved this for C++
Google is a place of conventions and consistency. It’s not surprising that the explicit error handling in Go (which is made by Google), is inspired deeply from how Google already does error handling in C++ (and possibly other languages).
Internally, Google heavily uses its (now open sourced) abseil
library
,
specifically - absl::StatusOr.
As a Xoogler myself, this is how I’d have written the code in Google/C++:
absl::StatusOr<int> CopyFile(string src, string dst) {
int numCopied = 0;
absl::Cleanup delOnErr([]() { numCopied == 0 && os.RemoveAll(dst); });
ASSIGN_OR_RETURN(File* r, os.Open(src));
absl::Cleanup rcloser([&]() { r.Close(); });
ASSIGN_OR_RETURN(File* w, os.Create(dst));
absl::Cleanup wcloser([&]() { w.Close(); });
ASSIGN_OR_RETURN(numCopied, io.Copy(w, r));
return n
}
Eagle eyed viewers would also notice absl::Cleanup being strikingly similar
to Go’s defer (I think that might have been the inspiration for Go devs!).
Note that this hypothetical example assumes that os.Open and os.Create return
absl::StatusOr<File*> and io.Copy returns absl::StatusOr<int>.
Here’s why this code looks much cleaner:
You can only return EITHER the return value OR an error. Which means instead of
writing return 0, err everywhere, you’d have just return return err.
Both T and absl::Status can be assigned to absl::StatusOr<T> without any
type casting, so there is no redundant casts - you just return what you intend.
C++ also has macros. In this case,
ASSIGN_OR_RETURN
actually encompasses the if-else logic Go is (in)famous for (you can find the
source in the linked tensorflow repo).
How Even solved this for Go
Go doesn’t have macros. This means that you can’t refactor out the repeated
“if-else” blocks into a separate function (since any return statement would
return from the refactored function but not the parent function).
One can mimic try-catch in Go using a combination of panic and recover. But
as mentioned earlier, that approach has its own downsides. Specifically, it
makes it very easy to forget about errors.
We decided to use a combination of panic and recover to create a library
that closely mimics the functionality from StatusOr.
Let’s say we define a function Check that panics if err != nil.
func Check(err error) {
if err != nil {
panic(err)
}
}
// usage:
n, err := io.Copy(src, dst)
Check(err)
This can help reduce some extra lines, but now we also need to handle the error. Using named function parameters, we can catch and re-assign the error to function’s error return.
func HandleErr(err *error) {
if e := recover(); e != nil {
*err = e.(error)
}
}
func CopyFile(src, dst *os.File) (_ int, err error) {
defer HandleErr(&err)
n, err := io.Copy(src, dst)
Check(err)
return n
}
Still not good enough. Recovering need not always give a valid error object.
We also probably don’t want to break the existing features that panic by
accidentally catching them. We don’t want to panic inside a recover :) We need
to introduce a custom error type.
type customError struct {error}
func (w wrappedError) Unwrap() error { return w.error }
func HandleErr(err *error) {
e := recover();
if e == nil {
return
}
var errTyped wrappedError
if eError, ok := e.(error); ok && errors.As(eError, &errTyped) {
*err = errTyped
return
}
panic(e)
}
With this, our implementation works. The resulting code is still pretty long though (however slightly cleaner):
func CopyFile(src, dst string) (_ int, e error) {
defer HandleErrFunc(func(err error) {
os.RemoveAll(dst)
e = err
})
r, err := os.Open(src)
Check(err)
defer r.Close()
w, err := os.Create(dst)
Check(err)
defer func() { Check(w.Close()) }()
return io.Copy(w, r)
}
Implementation of HandleErrFunc is left to the reader as an exercise (its
slightly different from Handle in the sense that the function should ideally be
called even if the error was returned using a return keyword, i.e. it needs
to check the value of the named error param too).
Fixing the last issue with Go1.18 type params
The code above is cleaner than what we started with, but its still long. Now, with Go1.18 release (on date of this article!) we can fix this issue.
func CheckAndAssign[T any](val T, err error) T {
Check(err)
return val
}
We at Even , started using Golang type params while it was still in beta, just to write these two lines of code. Here’s the difference it makes:
Final Code
func CopyFile(src, dst string) (_ int, e error) {
defer HandleErrFunc(func(err error) {
os.RemoveAll(dst)
e = err
})
r := CheckAndAssign(os.Open(src))
defer r.Close()
w := CheckAndAssign(os.Create(dst))
defer func() { Check(w.Close()) }()
return CheckAndAssign(io.Copy(w, r)), nil
}
Note that this also allows chaining of functions.
This solution solves almost all the problems described in Go2 error handling official proposal without the need of an additional language feature. Additional error wrapping can also be done by passing options to Check and CheckAndAssign, or in HandleErr.
Note of caution
When you set up a convention like this, people tend to be over generous in using these utility functions. There are cases when using these functions can accidentally propagate errors that ideally should’ve been handled inline.
One such example is the firestoreDoc.Get(ctx) function. This returns a
non-nil error when the document doesn’t exist. While most errors are related to
network, or transient behavior, this error almost always needs to be handled
explicitly.
It is important that programmers and reviewers are aware of the implications of using such conventions, and use them where it helps readability, not everywhere.
Bonus
The wrappedError implementation shown above is for the sake of the post. Our
implementation also does bookkeeping, and stores the code locations (using the
runtime package) of each place where Check is called. This lets us always
get a trace of where an error came from, and the originally thrown error is
still preserved since the wrappedError follows the conventions outlined in
https://go.dev/blog/go1.13-errors
.
But isn’t panicking an anti-pattern?
Yes.
You shouldn’t be panicking in code unless some fatal error happens … usually.
Panicking falls in the same bucket as using the unsafe library, or overusing
the reflect library.
Go already has libraries that take the stance of encapsulating a piece of
logic that can’t be represented by normal language features using otherwise
anti-patterns. For instance fmt or log package doesn’t restrict you to
passing variables that implement the Stringer interface. It uses reflection
heavily, but cleanly abstracts the code away from the usual use of the library.
Similarly, the gob package heavily uses the unsafe package to do
manipulations and casting that’d otherwise be very slow in Go.
The implementation we present here doesn’t require any users of the library to break out of patterns set by the Go community. It helps achieve cleaner code with the constructs currently available in the language.