Graceful stopping in Go
It’s rude to deploy early and often if deploys interrupt user requests so we build our Go services at Betable to stop gracefully without interrupting anyone. The idea is to stop listening, presumably so a new process can take over, and let all open connections respond to any in-progress requests before finally stopping service. Incidentally, we use goagain
to restart without even stopping listening but that’s beyond the scope of this article.
main.main
does four things: listens, makes a Service
and sends it into the background, blocks until signaled, and afterwards stops the service gracefully. Listening and signal handling are right out of the standard library playbook except that, to my great annoyance, you need a *net.TCPListener
or *net.TCPConn
and not merely a net.Listener
or net.Conn
to call SetDeadline
.
service.Serve(listener)
accepts connections and handles each one of them in its own goroutine. It sets a deadline so listener.AcceptTCP()
doesn’t block forever and between successive iterations checks to see if it should stop listening.
service.serve(conn)
reads and writes and likewise does so on a deadline. It sets a deadline so conn.Read(buf)
doesn’t block forever and between writing a response and reading the next request or after the conn.Read(buf)
deadline checks to see if it should close the connection.
The various goroutines decide to close connections and listeners if the service’s channel receives a value which, because nothing ever sends on this channel, is only possible after service.Stop()
has closed the channel. Recall the built-in close
function that causes channels to forevermore receive the zero-value of the channel’s element type.
The final piece of the puzzle is waiting for all goroutines to die which is implemented via the standard library’s sync.WaitGroup
.
https://gist.github.com/rcrowley/5474430 has it all: