Jump to Content

Andrew M McCall

Custom Golang Http Router With Middleware

An exploration of writing an http router from scracth with middleware capabilties.

Note: I modeled this router after the work of Ben Hoyt’s: Different approaches to HTTP routing In Go.

Who is this for? Myself really. I wanted to solidify my understanding of http routers so I could be better informed about choosing third party versus standard library implementations.

I would not recommend using this in production. This has been built as an educational exercise that I plan to use from time to time in my personal projects.

Link To Repo: Github.com

Why Build A Golang Router Library From Scratch?

Go has a standard library that has already solved this problem. There is an endless collection of third party vendor router libraries out there - all of them with great documentation and users to back them up. So why would anyone want to implement their own http router?

When I was learning javascript, specifically NodeJS, everyone just told me to use something called express.js. Express.js is a framework for making web applications that run using Node.js. As many people learning a new skill, I never questioned why or how Express worked. I just used it, made some projects and at the time, it was good enough for me.

This lead me to develop a knowledge gap. One that had always nagged me. Not understanding how a dependency works gives me anxiety about my app.
This isn’t a test writing lecture.
It is more of the principle that it is good have an understanding how how to build something and how it works.

This begs an interesting question: do I need to know how everything works? If I was a bike mechanic, would I care to know how every derailleur works,or does it only matter that I have the skill and knowledge to install them?

At the end of the day, all people care about is the result. In the example of the bike mechanic, the reason to know how each derailleur works can effect the decision making process. If I change derailleurs, how does it effect the rest of the system? Having some detailed knowledge about the inner workings can help an expert make better, safer choices.

Setting up A Basic Web Server App With Go

In golang, you can start a http server in about just a few lines of code:

package main
import (
"fmt"
"log"
"net/http"
"strings"
amrouter "github.com/elkcityhazard/am-router"
)
func main() {
rtr := amrouter.NewRouter()
rtr.PathToStaticDir = "/static"
rtr.AddRoute("GET", "/", http.HandlerFunc(homeHandler), homeMiddleWare)
rtr.AddRoute("GET", "(^/[\\w-]+)/?$", func(w http.ResponseWriter, r *http.Request) {
key := rtr.GetField(r, 0)
fmt.Fprint(w, key)
})
srv := &http.Server{
Addr: ":8080",
Handler: rtr,
}
fmt.Println("running")
if err := srv.ListenAndServe(); err != nil {
log.Panic(err)
}
}
func homeHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "home handler")
}
func homeMiddleWare(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
log.Println("I am the home middleware")
next.ServeHTTP(w, r)
})
}
func addTrailingSlash(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Println("add trailing slash")
if !strings.HasSuffix(r.URL.Path, "/") {
http.Redirect(w, r.WithContext(r.Context()), r.URL.Path+"/", http.StatusSeeOther)
return
}
next.ServeHTTP(w, r)
})
}
view raw main.go hosted with ❤ by GitHub
  1. The main function is creating a new instance of our router.
  2. I added in the path to static directory. In my implementation, this must be added, or else ServeHTTP will just return since it won’t be able to find a file. This is because, by default, we are using the base path as the path to static dir. Since it won’t find any static files in the dir, it will return.
  3. Next, we are adding some routes. This isn’t that important at the moment just know that routes and middleware need to be added for the router to work.
  4. Next, we are creating a new http server instance. The critical part here is the Handler property. Since amrouter satisfies the http.Handler interface (it has a receiver function called ServeHTTP that takes a http response writer and a pointer to an http request).
  5. Finally, we are starting the server via srv.ListenAndServe()

An http server gets spun up, starts listening for requests on the specified port, and then responds to those requests if it can find a handler that matches the request path. Each request path has a corresponding http method, and http handler.

The hint here is that we need to have one http handler that acts as switch board for all requests and routes them to their appropriate handler.

Creating An http router with middleware

Defining The Router and Route Types

A Router is made of of routes. In order to use a custom serve mux, we need to make sure that whatever we pass into &http.Server{Handler: %v} satisfies the http.Handler interface. This is because when using interfaces, anything that you define to perform those receiver methods can be passed in.

This is especially good because it helps us have loosely coupled code.

If you go to the Go documentation you can see the following information:

As you can see, the http Server Handler is just an http Handler. So as long as we define ServeHTTP(ResponseWriter, *Request) we can pass that in to our server.

type AMRouter struct {
	PathToStaticDir   string
	EmbeddedStaticDir embed.FS
	IsProduction      bool
	Routes            []AMRoute
	Middleware        []MiddleWareFunc
	GlobalMiddleware  []MiddleWareFunc
}

type AMRoute struct {
	Method     string
	Path       *regexp.Regexp
	Handler    http.Handler
	Middleware []MiddleWareFunc
}

func NewRouter() *AMRouter {

	return &AMRouter{
		Routes:           []AMRoute{},
		Middleware:       []MiddleWareFunc{},
		GlobalMiddleware: []MiddleWareFunc{},
	}
}

This is the setup for my custom router.

AMRouter is a struct that takes in routes which I have defined as type AMRoute. The AMRouter will also have the receiver method ServeHTTP that satisfies the http.handler interface.

At a high level, the router is going to:

Since we will be using regex to match the route keys, we can define an empty struct type as well as receiver method to extract route keys:

type CtxKey struct{}

func (rtr *AMRouter) GetField(r *http.Request, index int) string {
	fields := r.Context().Value(CtxKey{}).([]string)
	if len(fields) > 0 {
		if index > len(fields) {
			return ""
		}
		return fields[index]
	} else {
		return ""
	}
}

The GetField method extracts the route key from the request context. The fields are then cast to a slice of string which are than accessed via the index parameter.

If no value are found, we just return an empty string.

AddRoute Helper For Custom Go http Router


// MiddleWareFunc is an alias for func(http.Handler) http.Handler
type MiddleWareFunc func(http.Handler) http.Handler

// AddRoute takes a method, pattern, handler, and middleware and adds it to an instance of AMRouter.Routes
// It can return a regex compile error
func (rtr *AMRouter) AddRoute(method string, pattern string, handler http.HandlerFunc, mware ...MiddleWareFunc) error {

	var mwareToAdd = []MiddleWareFunc{}

	if len(mware) > 0 {

		for _, mw := range mware {
			mwareToAdd = append(mwareToAdd, mw)
		}

	}

	re, err := regexp.Compile("^" + pattern + "$")
	if err != nil {
		return err
	}
	rtr.Routes = append(rtr.Routes, AMRoute{
		Method:     method,
		Path:       re,
		Handler:    handler,
		Middleware: mwareToAdd,
	})

	return nil
}

I defined a MiddleWareFunc type. This is just an alias for a middleware handler. A middleware handler is just a regular function that accepts an http.Handler and returns one.

Handlers get passed in to AddRoute at the end as variadic. This is so we can handle the case of either having, or not having middleware.

The rest of the AddRoute function is pretty simple to follow:

We are just creating a new AMRoute populating it’s fields, and appending it to an already existing instance of AMRouter. I elected to use regexp.Compile instead of regexp.MustCompile so I could return an error in the event that the regex wasn’t parsable. So this function returns an error of either nil or some value, but the only place that an error is generated is regexp.Compile.

Writing Our Own ServeHTTP Function to satisfy the http.Handler Interface

To satisfy the http Handler interface, our struct needs to have a receiver method called ServeHTTP that accepts a ResponseWriter and a request. I elected to use go’s static file server built into the standard library to handle static file serving. That is why we need to define the path to the static directory earlier. I created a helper function called ServeStaticDirectory which short circuts the ServeHTTP function if the path to the request lives in the static directory.

For these requests, we simply create an http.Fileserver, strip the prefix from the request, and serve the files from their location.

ServeStaticDirectory returns a bool value. If the value is true, we are all done, if the value if false, we continue processing the request.

Next, I define allow []string which holds are allowed methods. This is useful for when the route pattern matches, but the method does not. We can return a custom not allowed response so folks know that they don’t have the write method.

if len(allow) > 0 {

		var customErrFunc http.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			w.Header().Set("Allow", strings.Join(allow, ", "))
			w.WriteHeader(405)
			err := errors.New("405 method not allowed")
			fmt.Fprint(w, err.Error())
		})

		customErrFunc = rtr.AddMiddlewareToHandler(customErrFunc, rtr.GlobalMiddleware...)
		customErrFunc.ServeHTTP(w, r)
		return

	} else {
		rtr.Custom404Handler(w, r)
		return
	}

Originally I used http.Error for this, but when I started adding in middleware, I realized I needed to customize this slightly to continue processing global middleware. I use the helper reicever function AddMiddlewareTOHandler to pass global middleware into the custom error handler. This just takes the current version of the handler, and updates it to be wrapped in the next middleware handler and returns it.

Matching The Request Path To A Route

This is just a basic loop logic. Ranging over each entry in AMRouter.Routes, we look for any matching regex patterns using FindStringSubmatch. If there is any matches, it returns a slice of the matches. Note that the first match is always the entire string here.

If len(matches) > 0 we continue processing the request:

  1. Check if the method is correct. If it is not, we bail out of the current indexed item.
  2. We extract the route parameter via context.WithValue.
  3. We create a handler that wraps the route handler. The context gets passed in with the request context.
  4. We check if the route has any middleware. If true, we range over the midleware and update the handler to be wrapped in the middleware. These get added in reverse order so they operate in the order they were added by the user.
  5. We do the same thing for any golobal middleware
  6. If everything matched, we serve the handler.
  7. If nothing matched, but allow has length, we return the http.MethodNotAllowed handler.
  8. Finally, if nothing comes up as a match, we return a 404 not found page.

Adding Global Middleware To Custom http Router

To make global middleware available to all routes, I created a receiver method called Use.

This just appends a MiddlewareFunc to the GlobalMiddleWare slice value of the AMRouter. Remember, Middleware is just a func that accepts and returns an http.handler.

func (rtr *AMRouter) Use(mw func(http.Handler) http.Handler) {
	rtr.GlobalMiddleware = append(rtr.GlobalMiddleware, mw)
}

Custom 404 Not Found Page

func (rtr *AMRouter) Custom404Handler(w http.ResponseWriter, r *http.Request) {
	notFoundHandler := http.NotFoundHandler()

	if len(rtr.GlobalMiddleware) > 0 {
		notFoundHandler = rtr.AddMiddlewareToHandler(notFoundHandler, rtr.GlobalMiddleware...)
	}

	notFoundHandler.ServeHTTP(w, r)
}

The Custom404Handler repeats the same logic as the Method Not Allowed Handler. We simply extend it to continue processing any middleware as needed.

package amrouter
import (
"context"
"embed"
"errors"
"fmt"
"net/http"
"regexp"
"strings"
)
type CtxKey struct{}
func (rtr *AMRouter) GetField(r *http.Request, index int) string {
fields := r.Context().Value(CtxKey{}).([]string)
fmt.Println(fields)
if len(fields) > 0 {
if index > len(fields) {
return ""
}
return fields[index]
} else {
return ""
}
}
type AMRouter struct {
PathToStaticDir string
EmbeddedStaticDir embed.FS
IsProduction bool
Routes []AMRoute
Middleware []MiddleWareFunc
GlobalMiddleware []MiddleWareFunc
}
func NewRouter() *AMRouter {
return &AMRouter{
Routes: []AMRoute{},
Middleware: []MiddleWareFunc{},
GlobalMiddleware: []MiddleWareFunc{},
}
}
type AMRoute struct {
Method string
Path *regexp.Regexp
Handler http.Handler
Middleware []MiddleWareFunc
}
// MiddleWareFunc is an alias for func(http.Handler) http.Handler
type MiddleWareFunc func(http.Handler) http.Handler
// AddRoute takes a method, pattern, handler, and middleware and adds it to an instance of AMRouter.Routes
// It can return a regex compile error
func (rtr *AMRouter) AddRoute(method string, pattern string, handler http.HandlerFunc, mware ...MiddleWareFunc) error {
var mwareToAdd = []MiddleWareFunc{}
if len(mware) > 0 {
for _, mw := range mware {
mwareToAdd = append(mwareToAdd, mw)
}
}
re, err := regexp.Compile("^" + pattern + "$")
if err != nil {
return err
}
rtr.Routes = append(rtr.Routes, AMRoute{
Method: method,
Path: re,
Handler: handler,
Middleware: mwareToAdd,
})
return nil
}
func (rtr *AMRouter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Don't create new context unnecessarily
isStatic := rtr.ServeStaticDirectory(w, r)
if isStatic {
return
}
var allow []string
for _, route := range rtr.Routes {
matches := route.Path.FindStringSubmatch(r.URL.Path)
if len(matches) > 0 {
if r.Method != route.Method {
allow = append(allow, route.Method)
continue
}
// Store route parameters in context if needed
ctx := context.WithValue(r.Context(), CtxKey{}, matches[1:])
var handler http.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
route.Handler.ServeHTTP(w, r.WithContext(ctx))
})
// middleware gets handled outside in, so add route based first, then global
if len(route.Middleware) > 0 {
handler = rtr.AddMiddlewareToHandler(handler, route.Middleware...)
}
if len(rtr.GlobalMiddleware) > 0 {
handler = rtr.AddMiddlewareToHandler(handler, rtr.GlobalMiddleware...)
}
handler.ServeHTTP(w, r)
return
}
}
if len(allow) > 0 {
var customErrFunc http.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Allow", strings.Join(allow, ", "))
w.WriteHeader(405)
err := errors.New("405 method not allowed")
fmt.Fprint(w, err.Error())
})
customErrFunc = rtr.AddMiddlewareToHandler(customErrFunc, rtr.GlobalMiddleware...)
customErrFunc.ServeHTTP(w, r)
return
} else {
rtr.Custom404Handler(w, r)
return
}
}
// ServeStaticDirectory accepts an http.ResponseWriter, and a *http.Request and determins if
// the current r.URL.Path is to a static file. It returns a bool to indicate if the rest of the
// ServeHTTP function shoulbe be short circuited
func (rtr *AMRouter) ServeStaticDirectory(w http.ResponseWriter, r *http.Request) bool {
// handle static directory
if strings.HasPrefix(r.URL.Path, rtr.PathToStaticDir) {
// if not in prod, load static resources from disk, else embed
if !rtr.IsProduction {
fileServer := http.FileServer(http.Dir(rtr.PathToStaticDir))
http.StripPrefix("/static/", fileServer).ServeHTTP(w, r)
} else {
fileServer := http.FileServer(http.FS(rtr.EmbeddedStaticDir))
http.StripPrefix("/static/", fileServer).ServeHTTP(w, r)
}
return true
}
return false
}
// Use adds global middleware to all routes
func (rtr *AMRouter) Use(mw func(http.Handler) http.Handler) {
rtr.GlobalMiddleware = append(rtr.GlobalMiddleware, mw)
}
// AddMiddlewareToHandler applies middleware in reverse order
func (rtr *AMRouter) AddMiddlewareToHandler(handler http.Handler, middleware ...MiddleWareFunc) http.Handler {
// Apply middleware in reverse order to maintain correct execution order
for i := len(middleware) - 1; i >= 0; i-- {
currentMiddleware := middleware[i]
handler = currentMiddleware(handler)
}
return handler
}
func (rtr *AMRouter) Custom404Handler(w http.ResponseWriter, r *http.Request) {
notFoundHandler := http.NotFoundHandler()
if len(rtr.GlobalMiddleware) > 0 {
notFoundHandler = rtr.AddMiddlewareToHandler(notFoundHandler, rtr.GlobalMiddleware...)
}
notFoundHandler.ServeHTTP(w, r)
}
view raw am_router.go hosted with ❤ by GitHub
Tags: