Patterns for scalable web services in Go

Hi, I’m Richard

r@rcrowley.org or @rcrowley
2007 Wash U grad

Betable

Gambling-as-a-Service
Licensed and regulated so game developers don’t have to be

Scalability

Of human resources
Of machine resources

Polls

Who has written at least one Go program?
Who has Go in production?
Who has only Go in production?

Good reasons people choose Go

Brevity
Static type system
Compiles to x86, ARM, etc.
CSP
Thompson, Pike, Cox, Griesemer
Brad Fitz

Bad reasons people choose Go

It isn’t Java
It isn’t running on the JVM
(Guilty)

Good reasons to not choose Go

Your problem won’t tolerate a naive garbage collector
Library immaturity overburdens your team

Rule #1

Always google it as “golang”

Hello, www!

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc(
        "/",
        func(w http.ResponseWriter, r *http.Request) {
            fmt.Fprintln(w, "Hello, www!")
        },
    )
    http.ListenAndServe(":8080", nil)
}

Functions

func handler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/plain")
    w.WriteHeader(http.StatusOK)
    fmt.Fprintln(w, "Hello, www!")
})

May be named or anonymous
All functions are closures
Zero or more typed parameters
Parameters are pass-by-value
Zero or more typed return values

Errors

func fail() error { return errors.New("fail") }
func multifail() (*Win, error) {
    return nil, errors.New("multifail")
}

Conventionally return an error last
There are no exceptions because errors are not exceptional
Use error to communicate what happened to the caller

Structs

type handler struct {
    counter metrics.Counter
}

type Authorization struct {
    Username, Password string
}

Structs are types
Zero or more typed fields

Methods

func (h *handler) ServeHTTP(
    w http.ResponseWriter,
    r *http.Request,
) {
    w.Header().Set("Content-Type", "text/plain")
    w.WriteHeader(http.StatusOK)
    fmt.Fprintln(w, "Hello, www!")
    h.counter.Inc(1)
}

Method receiver may be any type in the declaring package

Interfaces

// In the standard library's io/io.go:
type Reader interface {
    Read(p []byte) (n int, err error)
}

// In the standard library's net/http/server.go:
type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

Interfaces are also types
Zero or more method signatures
Types that implement all methods implement the interface implicitly

More realistic HTTP

func main() {
    mux := http.NewServeMux()
    mux.Handle("/", &handler{})
    server := &http.Server{Handler: mux}
    listener, err := net.Listen("tcp", ":8080")
    if nil != err {
        log.Fatalln(err)
    }
    if err := server.Serve(listener); nil != err {
        log.Fatalln(err)
    }
}

http.ServeMux is another http.Handler

Under the hood

Raise your hand if you’ve been burned by BaseHTTPServer or WEBrick
What’s Serve do, anyway?
(For one thing, power dl.google.com)

Echo server

func main() {
    listener, err := net.Listen("tcp", ":1234")
    if nil != err {
        log.Fatalln(err)
    }
    for {
        conn, err := listener.Accept()
        if nil != err {
            log.Fatalln(err)
        }
        go handle(conn)
    }
}

go makes any function call asynchronous in a “goroutine”

Echo server

func handle(conn net.Conn) {
    defer conn.Close()
    p := make([]byte, 4096)
    for {
        n, err := conn.Read(p)
        if nil != err {
            log.Println(conn.RemoteAddr(), err)
            break
        }
        log.Printf("%v p: %s", conn.RemoteAddr(), p[:n])
        if _, err := conn.Write(p[:n]); nil != err {
            log.Println(conn.RemoteAddr(), err)
            break
        }
    }
}

No callbacks, no miniature state machines

Goroutine per connection

Many goroutines are scheduled onto GOMAXPROCS operating system threads
Cheap enough to not worry about pooling
This is how the standard library HTTP server works, too

CSP

Tony Hoare’s Communicating Sequential Processes in CACM volume 21 issue 8
Go’s summary: “Do not communicate by sharing memory; instead, share memory by communicating.”
Useful for connection handling, critical sections, and pooled operations

Goroutines

Been there, done that
Hoare’s “Parallel Commands”

Channels

ch := make(chan int) // unbuffered
ch := make(chan int, 1) // buffer length of 1
ch <- 47 // send
<-ch // receive

Buffered (asynchronous + backpressure) or unbuffered (synchronous)
Concurrent send and receive operations
Faster than mutexes
Hoare’s “Input and Output Commands”

Beware CSP in OLTP

Sacrificing shared nothing in the small precludes diagonal scaling
Non-empty channel buffers are an opportunity for data loss

Graceful stop

The least surprising opportunity to screw things up
http package is annoyingly unhelpful
Not being addressed in Go 1.2 so we take matters into our own hands

Clumsy stop for HTTP

func main() {
    mux := http.NewServeMux()
    mux.Handle("/", &handler{})
    server := &http.Server{Handler: mux}
    listener, err := net.Listen("tcp", ":1234")
    if nil != err {
        log.Fatalln(err)
    }
    ch := make(chan os.Signal)
    signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM)
    go server.Serve(listener)
    log.Println(<-ch)
    listener.Close()
    time.Sleep(60e9) // FIXME!
}

Zero-downtime restart

func main() {
    // mux ... server ...
    listener, ppid, err := goagain.GetEnvs()
    if nil != err {
        listener, _ = net.Listen("tcp", ":8080")
        go server.Serve(listener)
    } else {
        go server.Serve(listener)
        goagain.KillParent(ppid)
    }
    goagain.AwaitSignals(listener)
    listener.Close()
    time.Sleep(60e9) // FIXME!
}

goagain handles SIGUSR2
Difficult to supervise; systemd > Upstart

Higher-level frameworks

Gorilla, Revel: more ambitious and all-knowing; appropriate for web applications
gorest: hides a lot in struct tags
pat: unopinionated but leaves most concerns unaddressed
Many others we can talk about over beers

Tiger Tonic

Tiger Tonic

Newly open-source!
Inspired by Dropwizard
How Betable builds web services in Go
github.com/rcrowley/go-tigertonic

Routing

mux := http.NewServeMux()
mux.Handle("/", &handler{})

http.ServeMux only routes prefixes

mux := tigertonic.NewTrieServeMux()
mux.Handle("GET", "/foo/{bar}/baz", &handler{})

tigertonic.TrieServeMux is method- and wildcard-aware
Responds 404 and 405 appropriately

JSON

var handler http.Handler = tigertonic.Marshaled(func(
    url.URL, http.Header, *MyRequest,
) (int, http.Header, *MyResponse, error) {
    return http.StatusOK, nil, &MyResponse{}, nil
})

Static request and response types
No hassling with json.Encoder
Responds 400, 406, and 415 appropriately
Responds with JSON in case of error, too
Still an http.Handler

Reflection underneath

decoder := reflect.ValueOf(json.NewDecoder(r.Body))
out := decoder.MethodByName("Decode").Call([]reflect.Value{
    reflect.New(m.v.Type().In(2).Elem()),
})

User code deals only with static types

if !t.Out(2).Implements(reflect.TypeOf((*Response)(nil)).Elem()) {
    panic("type ... was %v, not Response", t.Out(2))
}

Runtime enforces fuzzy interfaces

Conditionals

handler = tigertonic.If(
    func(r *http.Request) (http.Header, error) {
        if "" == r.Header.Get("X-Condition") {
            return nil, Forbidden{errors.New("forbidden")}
        }
        return nil, nil
    }),
    handler, // from two slides ago
)

If naturally wraps any http.Handler
Special case of First middleware chains
And yes, it’s an http.Handler

Visibility

Logging via the standard library
Metric collection via go-metrics
http.Handler all the way down

Logging

handler = tigertonic.Logged(handler, nil)

tigertonic.Logged for full (optionally redacted) request and response logs

handler = tigertonic.ApacheLogged(handler)

tigertonic.ApacheLogged for Apache combined logs

Metrics

handler = tigertonic.Counted(handler, "my-handler", nil)
handler = tigertonic.Timed(handler, "my-handler", nil)

tigertonic.Counted and tigertonic.Timed for knowing how many and how fast
We mostly use tigertonic.Timed on each route individually

Other batteries included

Strongly-typed per-request context
Middleware chains
URL namespaces
Virtual hostnames
HTTP Basic auth
CORS basics
TLS

Testing

Construct a fake http.Request
Call ServeHTTP
Use httptest.ResponseRecorder
Call your Marshaled function directly

Dependency hell

go get tool invites you right in

Dependency purgatory

Go-specific solutions:
gopack and johnny-deps
git-submodule is perfectly fine
Google vendors third-party code
We vendor third-party code as part of CI

Reverse proxy

TLS termination in Tiger Tonic
No crypto to block Nginx event loop
One fewer hops means lower latency

TLS in Go

c, _ := tls.LoadX509KeyPair(certPathname, keyPathname)
listener, err := tls.Listen("tcp", ":4443", &tls.Config{
    Certificates: []tls.Certificate{c},
    CipherSuites: []uint16{tls.TLS_RSA_WITH_RC4_128_SHA},
})

Always use net.Conn and net.Listener interfaces to serve either TLS and non-TLS
Replace crypto/tls package with openssl binding as an optimization

Waste not

Favor local variables that don’t “escape” onto the heap
Reuse allocated buffers
A small heap is a fast heap

Small pieces, loosely joined

Use the standard library
Implement interfaces when you can
Fail fast and with details

Share nothing

Accepted in the large
Holds true in the small within a Go process

Concurrency primitives

Goroutine-per-connection
Limit concurrency when necessary: one channel, m senders, and n receivers

Measure everything

Macro statistics from Counted and Timed
Micro statistics from runtime/pprof and go tool pprof

Parting advice

Use http.Handler for web services
Use concurrency primitives for concurrent or pipelined tasks
Bridge the gap cautiously
Remember that errors are not exceptional

Open-source goodness

github.com/d2fn/gopack
github.com/rcrowley/goagain
github.com/rcrowley/go-metrics
github.com/rcrowley/go-tigertonic
github.com/VividCortex/johnny-deps

A word from my sponsors

jobs@betable.com

Thank you