Golang is a fantastic programming language with so many features supported out of the box. One such feature is a full-fledged HTTP server, which requires only a few lines of code to start. However, not many people are aware of the mechanism to gracefully shut down the server. In fact even Golang's official HTTP server tutorial does not mention this important feature. In this blog post we’ll explore a problem with the hello world version of the HTTP server in Golang, and how graceful shutdown addresses it.
The basic HTTP server
The most basic HTTP server can be illustrated by the following example.
package main
import (
"log"
"net/http"
)
func main() {
http.HandleFunc("/hello", func(w http.ResponseWriter, req *http.Request) {
w.Write([]byte(`hello world`))
})
if err := http.ListenAndServe("localhost:8081", nil); err != nil {
log.Println("http server error:", err)
}
}
To run the example above, we can copy and paste it into a main.go file within a folder of our choice, e.g. ~/go-server-demo/main.go, then run GO111MODULE=off go run . provided that Golang is already installed in our local machine.
In the example we provide an HTTP handler for the default server mux, which returns the text hello world whenever there’s a request to http://localhost:8081/hello. Note that the function http.ListenAndServe will block until an error is returned from our server. For a beginner friendly tutorial this is sufficient, but it is not production ready. The reason is pretty simple: there’s no handling of termination signals. A SIGINT, for example, sent to the server will shut it down abruptly with no room for inflight HTTP requests to finish. In our local machine, a SIGINT can be invoked by pressing CTRL + C.
The following diagram explains the problem better.
As can be seen in the diagram, request number #3 and #4 will not be handled correctly because they’re still being processed when the server shuts down. Client A and Client B will receive a 5xx HTTP status code for request 3 and request 4 respectively.
HTTP Server with graceful shutdown
Fortunately Golang HTTP server provides a native shutdown function. It must be noted that implementing a shutdown mechanism requires basic understanding of Golang concurrency, which is not covered in this post.
Firstly we need to handle the receiving of termination signals. In Golang this can be done quite easily with channel.
// create a buffered channel to avoid blocking sender.
c := make(chan os.Signal, 1)
// relay SIGINT and SIGTERM to c.
// SIGTERM is included here, because it's used by some
// infrastructure orchestrator like Kubernetes
// to communicate with its containers.
signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
Then we need to “listen” to the value sent to c, and shut down the server when it happens. This can be done with a select statement.
select {
case <-c:
// shutdown the server gracefully
}
Another approach, which is probably more idiomatic, is to use signal.NotifyContext to mark a context as done when a termination signal is received.
// create a background context to listen to termination signals.
ctx := make(context.Background())
// when SIGINT or SIGTERM is sent by the host machine
// the done channel of ctx will be closed.
ctx, _ = signal.NotifyContext(ctx, syscall.SIGINT, syscall.SIGTERM)
select {
case <-ctx.Done():
// shutdown the server gracefully
}
For the remaining parts of this post, we’ll go with the first approach because it’s easier to reason about for Golang beginners.
The next step is the interesting part. In order to invoke the method to shut down the server gracefully, some modifications must be made in the basic implementation. An http.Server, which implements a method Shutdown, must be constructed and then its method ListenAndServe is called instead of using http.ListenAndServe function.
http.HandleFunc("/hello", func(w http.ResponseWriter, req *http.Request) {
w.Write([]byte(`hello world`))
})
s := http.Server{Addr: "localhost:8081"}
if err := s.ListenAndServe(); err != nil {
log.Println("http server error:", err)
}
Now that we have a http.Server we can call its Shutdown method when receiving SIGINT or SIGTERM.
ctx := make(context.Background())
signal.NotifyContext(ctx, syscall.SIGINT, syscall.SIGTERM)
select {
case <-c:
log.Println("prepare to shutdown server")
if err := s.Shutdown(context.Background()); err != nil {
log.Fatal("server shutdown failed:", err)
}
log.Println("server shutdown completed.")
}
What does the Shutdown method do? Its documentation articulates perfectly.
// Shutdown gracefully shuts down the server without interrupting any
// active connections. Shutdown works by first closing all open
// listeners, then closing all idle connections, and then waiting
// indefinitely for connections to return to idle and then shut down.
// If the provided context expires before the shutdown is complete,
// Shutdown returns the context's error, otherwise it returns any
// error returned from closing the Server's underlying Listener(s).
However this is not enough because s.ListenAndServe still blocks until there’s an error returned. Appending the above code to handle termination signal after s.ListenAndServe will not work.
// WILL NOT WORK because s.ListenAndServe will block forever until an error happens to the server.
if err := s.ListenAndServe(); err != nil {
log.Println("HTTP server error:", err)
}
ctx := make(context.Background())
signal.NotifyContext(ctx, syscall.SIGINT, syscall.SIGTERM)
select {
case <-c:
log.Println("prepare to shutdown server")
if err := s.Shutdown(context.Background()); err != nil {
log.Fatal("server shutdown failed:", err)
}
log.Println("server shutdown completed.")
}
We have to use a goroutine here.
go func() {
if err := s.ListenAndServe(); err != nil {
if !errors.Is(err, http.ErrServerClosed) {
log.Println("HTTP server error:", err)
}
}
}()
ctx := make(context.Background())
signal.NotifyContext(ctx, syscall.SIGINT, syscall.SIGTERM)
select {
case <-c:
log.Println("prepare to shutdown server")
if err := s.Shutdown(context.Background()); err != nil {
log.Fatal("server shutdown failed:", err)
}
log.Println("server shutdown completed.")
}
By calling s.ListenAndServe in a goroutine, our main goroutine is not blocked and termination signals can be listened to and handled correctly. We add another condition check errors.Is(err, http.ErrServerClosed) because after calling s.Shutdown the error http.ErrServerClosed will be returned from s.ListenAndServe, and it’s not really an error. This part could be a bit confusing, but Golang errors can actually be used to denote a indication to stop a process. We got a few examples in the standard packages, io.EOF denoting the end of file, for instance.
One must be aware that wrapping the part handling termination signals in a goroutine while letting s.ListenAndServe occupy the main goroutine will not work.
// WILL NOT WORK
go func() {
c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
select {
case <-c:
log.Println("prepare to shutdown server")
if err := s.Shutdown(context.Background()); err != nil {
log.Fatal("server shutdown failed:", err)
}
log.Println("server shutdown completed.")
}
}()
if err := s.ListenAndServe(); err != nil {
if !errors.Is(err, http.ErrServerClosed) {
log.Println("HTTP server error:", err)
}
}
Why? Because when Shutdown is called, ListenAndServe will immmediately return ErrServerClosed, which causes our main goroutine to exit immediately while inflight requests are still being served. And if the main goroutine exits, then all other goroutines also exit. Therefore, we have to let our main goroutine wait on Shutdown instead of ListenAndServe.
Now we have a working HTTP server that gracefully shuts down. There is a minor improvement that we can implement. By default Shutdown will wait indefinitely until all inflight requests are completed, and it may not be a desired behavior. A context with a timeout value could be used here to force the server to shut down after a certain duration.
select {
case <-c:
log.Println("prepare to shutdown server")
// wait 15 seconds for inflight requests to complete before forcing the server to shutdown
ctx, cancelFn := context.WithTimeout(context.Background(), 15*time.Second)
defer cancelFn()
if err := s.Shutdown(ctx); err != nil {
log.Fatal("server shutdown failed:", err)
}
log.Println("server shutdown completed.")
}
This is important because some orchestration tool like Kubernetes only gives our containers 30 seconds to shut down by default. At this point we finally reach a production-ready HTTP server that handles shutdown appropriately. It’s possible to conduct some kind of experiment to see the difference between the default way and the graceful way of constructing an HTTP server, but we’ll leave that for another blog post. The source code used in this article can be found here.
Beyond HTTP server
So far we have gone through an implementation of graceful shutdown for an HTTP server in Golang. However, Golang applications can be more than that. An application can be a subscriber to consume messages from a message queue, or it could a background application that runs according to a cron schedule, for example.
Whatever application type we’re working with, mishandling signal could lead to critial defects: a web server could halt during processing a request that’s involved in multiple steps, a background data processor could leave the data set in an unsuable state if halted abrubtly. Graceful shutdown allows us to intercept signals, finish whatever our application is doing, as well as closing opened connections and reclaiming allocated resources. It should not be neglected.