Integrarting Naurt’s Geocoder with Go

Setting up the Environment with Go

If you haven’t yet, make sure to install Go. You can follow the official instructions here, but on ArchLinux you can install with

$ sudo pacman -S go

And then add Go to your PATH with

export PATH=$PATH:/usr/local/go/bin

In either your .zshrc or .bashrc, whichever you use. Confirm your Go installation with

$ go version

To set up the project itself, let’s create a folder, initialise a module, and create our source files.

$ mkdir naurt_example
$ go mod init naurt_example
$ mkdir src
$ touch src/main.go

A Simple Webserver

We can place the following boilerplate in main.go

package main

import (
	"fmt"
	"net/http"
)

func main() {

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Hello world!")
	})

	fmt.Println("Starting the server!")
	if err := http.ListenAndServe(":8080", nil); err != nil {
		fmt.Println((err))
	}
}

And then run with

$ go build -C src -o naurt
$ ./src/naurt

If you use Windows, you will need to use the following

$ go build -C src -o naurt.exe
$ src/naurt.exe

You should see the web server starting, and if you go to localhost:8080/ in a web browser see a hello world message

Using a Naurt API Key

Using Naurt services requires an API Key. If you don’t already have one, you can sign up to the dashboard for free. We don’t require a credit card. You’ll get a free key loaded with thousands of requests. I’ll place my api key in a file called api.key next to the go.mod file.

In Go, we’ll read the key in as a constant from a file. You’ll need to import os , io and sync. Start by creating the apiKey object

var (
	apiKey string
	once   sync.Once
)

func initialiseApiKey() {
	// Use sync.Once to ensure this is executed only once
	once.Do(func() {
		// Read file content from a file
		file, err := os.Open("api.key")
		if err != nil {
			panic("`api.key` not found")
		}

		defer file.Close()

		content, err := io.ReadAll(file)
		if err != nil {
			panic("Could not read API key to string")
		}

		apiKey = string(content)
	})
}

And then we need to call this in the main function like so

func main() {

	initialiseApiKey()

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Hello world!")
	})

	fmt.Println("Starting the server!")
	if err := http.ListenAndServe(":8080", nil); err != nil {
		fmt.Println((err))
	}
}

apiKey is now globally available across threads to be used.

Creating a Naurt Request

Create a new file called types.go in the src folder - we’ll put all of the structs which handle the Naurt request in here.

$ touch src/types.go

We’ll start by typing the Naurt request, so the file only needs the following

package main

type NaurtRequest struct {
	AddressString     string   `json:"address_string,omitempty"`
	Latitude          *float64 `json:"latitude,omitempty"`
	Longitude         *float64 `json:"longitude,omitempty"`
	AdditionalMatches bool     `json:"additional_matches,omitempty"`
}

Now that we have this type, we can create a simple function which will make this request for us. For now, our goal is to simply make the request and print out the JSON body we get back. We’ll define a function makeNaurtRequest which will handle actually making a request to Naurt. For now, it’ll be simple and we’ll just hard code the request in, but we can come back and customise that later on.

func makeNaurtRequest() (string, error) {
	url := "https://api.naurt.net/final-destination/v1"

	data := NaurtRequest{
		AddressString: "The Grand Hotel, Brighton",
	}

	jsonData, err := json.Marshal(data)
	if err != nil {
		return "", err
	}

	req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonData))
	if err != nil {
		return "", err
	}

	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("Authorization", apiKey)

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return "", err
	}

	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", err
	}

	return string(body), err
}

It will also be easier if we write a dedicated handler function

func handler(w http.ResponseWriter, r *http.Request) {
	resp, err := makeNaurtRequest()
	if err != nil {
		fmt.Fprintf(w, err.Error())
		return
	}

	fmt.Fprintf(w, resp)

}

And then finally we can update the main function to use this new handler

func main() {

	initialiseApiKey()

	http.HandleFunc("/", handler)

	fmt.Println("Starting the server!")
	if err := http.ListenAndServe(":8080", nil); err != nil {
		fmt.Println((err))
	}
}

Go ahead and compile and run the code again - you should see a JSON print out when you go to localhost:8080/

Creating a Naurt Response

If we want to take this further, we’ll need to convert the JSON response we get from Naurt into data that we can work with more effectively in Go. To do this, we’ll use structs like the NaurtRequest to deserialise the response into. I’ll put the following types into types.go

type NaurtResponse struct {
	BestMatch         *DestinationResponse   `json:"best_match,omitempty"`
	AdditionalMatches *[]DestinationResponse `json:"additional_matches,omitempty"`
	Version           string                 `json:"version,omitempty"`
}

type DestinationResponse struct {
	ID       string       `json:"id"`
	Address  string       `json:"address"`
	Geojson  NaurtGeojson `json:"geojson"`
	Distance float32      `json:"distance,omitempty"`
}

type NaurtGeojson struct {
	Features []Feature `json:"features"`
	TypeVal  string    `json:"type"`
}

type Feature struct {
	Geometry   Coordinates `json:"geometry"`
	TypeVal    string      `json:"type"`
	Properties Properties  `json:"properties"`
}

type Coordinates struct {
	Coordinates CoordinatesWrapper `json:"coordinates"`
	TypeVal     string             `json:"type"`
}

type CoordinatesWrapper struct {
	Number       [][]float32
	NestedNumber [][][]float32
}

type Properties struct {
	NaurtType    string   `json:"naurt_type"`
	Contributors []string `json:"contributors"`
}

This is a fairly straightforward JSON deserialisation task, however there’s a few points worth looking at in more detail. I called all the type fields from the JSON TypeVal  since type is already a keyword in Go. Thankfully, Go makes it really easy to specify the name in the JSON.

The other point is CoordinatesWrapper - notice that it does not have a json marking. This is because Naurt can respond with two kinds of coordinates

1. A double nested array. This represents a multipoint type, for example, the naurt_doors will be of this type.
2. A triple nested array. This represents a multipolygon type, for example, the naurt_parking and naurt_building are of this type

However, both will not be present at once. Therefore, we need to write a small piece of custom deserlisation code for this

func (f *CoordinatesWrapper) UnmarshalJSON(data []byte) error {

	var doubleArray [][]float32
	if err := json.Unmarshal(data, &doubleArray); err == nil {
		f.Number = doubleArray
		return nil
	}

	var tripleArray [][][]float32
	if err := json.Unmarshal(data, &tripleArray); err == nil {
		f.NestedNumber = tripleArray
		return nil
	}

	return errors.New("`CoordinatesWrapper` did not find valid format")
}

Note that you will need to import encoding/json and errors .

The point of this custom deserialise code is the struct will either have a double nested array or a triple nested array, while the other is nil and we can check for this at run time.

Now, we’ll edit the makeNaurtRequest function to return a NaurtRequest rather than a string

func makeNaurtRequest() (NaurtResponse, error) {
	url := "https://api.naurt.net/final-destination/v1"

	data := NaurtRequest{
		AddressString: "The Grand Hotel, Brighton",
	}

	jsonData, err := json.Marshal(data)
	if err != nil {
		return NaurtResponse{}, err
	}

	req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonData))
	if err != nil {
		return NaurtResponse{}, err
	}

	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("Authorization", apiKey)

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return NaurtResponse{}, err
	}

	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return NaurtResponse{}, err
	}

	var naurt NaurtResponse
	if err := json.Unmarshal([]byte(body), &naurt); err != nil {
		return NaurtResponse{}, err
	}

	return naurt, err
}

We’ll also temporarily update the handler function to deal with this new return type. For now, we’ll just convert it back into a JSON and print it out. Although it seems like a lot of work to do nothing, we will very soon do something much more useful with the response

func handler(w http.ResponseWriter, r *http.Request) {
	resp, err := makeNaurtRequest()
	if err != nil {
		fmt.Fprintf(w, err.Error())
		return
	}

	jsonBody, err := json.Marshal(resp)
	if err != nil {
		fmt.Fprintf(w, err.Error())
		return
	}

	fmt.Fprintf(w, string(jsonBody))

}

You should see the JSON again, but this time it has first been processed as a Go object we can work with.

Using URL Arguments

At the moment, we’ve hardcoded the request into the program. This is obviously much less useful than having the request be dynamic. The design we’ll use is parsing arguments out of the URL. Editing the makeNaurtRequest function is really straightforward

func makeNaurtRequest(address string, latitude *float64, longitude *float64) (NaurtResponse, error) {
	url := "https://api.naurt.net/final-destination/v1"

	data := NaurtRequest{
		AddressString:     address,
		Latitude:          latitude,
		Longitude:         longitude,
		AdditionalMatches: true,
	}
	...

Make sure you import strcnov, then we can edit the handler to look like so

func handler(w http.ResponseWriter, r *http.Request) {

	query := r.URL.Query()

	address := query.Get("address")

	var latitude *float64
	if query.Has("latitude") {
		tmp, err := strconv.ParseFloat(query.Get("latitude"), 64)
		if err != nil {
			fmt.Fprintf(w, err.Error())
			return
		}
		latitude = &tmp
	}

	var longitude *float64
	if query.Has("longitude") {
		tmp, err := strconv.ParseFloat(query.Get("longitude"), 64)
		if err != nil {
			fmt.Fprintf(w, err.Error())
			return
		}
		longitude = &tmp
	}

	resp, err := makeNaurtRequest(address, latitude, longitude)
	if err != nil {
		fmt.Fprintf(w, err.Error())
		return
	}

	jsonBody, err := json.Marshal(resp)
	if err != nil {
		fmt.Fprintf(w, err.Error())
		return
	}

	fmt.Fprintf(w, string(jsonBody))

}

Since address is meant to be a string we can just get it from the query - if it isn’t present we will simply get the empty string, which the JSON serialisation will ignore. However, we can now see why the latitude and longitude have been *float64 so far. If these arguments are not provided, we need them to be an empty type so they won’t serialise. If we can’t parse the arguments given in latitude or longitude as floats, we return an error, though.

Now if you compile and run the code, and use a search query, you’ll get different results! For example, try http://localhost:8080/?address=The Grand Hotel, Eastbourne

Plotting Naurt on a Map

At this point, if we were using Naurt purely in a backend application we’d be more or less done - this could be used in a route planning or ETA system as is. However, we’ll plot the output on a map just so we can visually see the results.

To do the plotting, we’ll make a new file called plotting.go

$ touch src/plotting.go

For this application we’ll be using Plotly, and there’s actually already a Plotly package in Go. We’ll need to install it though, which we can do with

$ go get github.com/MetalBlueberry/go-plotly

Inside plotting.go make sure to add package main to the top.

We’ll start by making some helper functions. Plotly typically expects all the latitudes to be grouped into one slice and all the longitudes grouped into another slice. Naurt follows the geojson standard, so the latitudes and longitudes are grouped into a single slice representing a point. We can create the following very simple functions to convert between the two

func extractLats(points [][]float32) []float32 {
	lats := []float32{}

	for _, point := range points {
		lats = append(lats, point[1])
	}

	return lats
}

func extractLons(points [][]float32) []float32 {
	lons := []float32{}

	for _, point := range points {
		lons = append(lons, point[0])
	}

	return lons
}

All of the useful data to do with plotting is found inside the DestinationResponse struct, so let’s write a function which can convert the DestinationResponse into something Plotly can work with.

func extractNaurtInner(data *DestinationResponse, traces *[]grob.Trace, layout **grob.Layout, bestMatch bool) {
	for _, feat := range data.Geojson.Features {
		if feat.Geometry.Coordinates.Number != nil {
			// naurt_door

			if bestMatch {
				*layout = &grob.Layout{
					Mapbox: &grob.LayoutMapbox{
						Style: "carto-positron",
						Center: &grob.LayoutMapboxCenter{
							Lat: float64(feat.Geometry.Coordinates.Number[0][1]),
							Lon: float64(feat.Geometry.Coordinates.Number[0][0]),
						},
						Zoom: 15.0,
					},
					Showlegend: grob.False,
					Autosize:   grob.True,
				}
			}

			trace := &grob.Scattermapbox{
				Type:   "scattermapbox",
				Lat:    extractLats(feat.Geometry.Coordinates.Number),
				Lon:    extractLons(feat.Geometry.Coordinates.Number),
				Text:   fmt.Sprintf("%s - %s", data.Address, feat.Properties.NaurtType),
				Marker: &grob.ScattermapboxMarker{Size: 9.0},
			}

			*traces = append(*traces, trace)

		} else if feat.Geometry.Coordinates.NestedNumber != nil {
			// naurt_parking or naurt_building

			for _, shape := range feat.Geometry.Coordinates.NestedNumber {

				trace := &grob.Scattermapbox{
					Type: "scattermapbox",
					Lat:  extractLats(shape),
					Lon:  extractLons(shape),
					Text: fmt.Sprintf("%s - %s", data.Address, feat.Properties.NaurtType),
					Mode: grob.ScattermapboxModeLines,
					Fill: grob.ScattermapboxFillToself,
				}

				*traces = append(*traces, trace)
			}
		}
	}
}

Make sure you import grob "[github.com/MetalBlueberry/go-plotly/graph_objects](<http://github.com/MetalBlueberry/go-plotly/graph_objects>)"  as well as "fmt" .

Plotly expects a slice of traces, so in the extractNaurtInner function, we pass in a reference to a slice of these traces that we’ll make outside the function and then continually append to. We also need a layout for a Plotly map. The layout contains the map centre. We’ll centre on the door of the best match from Naurt, which is why we need to tell this function whether it’s a best match or not. Again, we’ll pass in a pointer to this layout object and set it inside the function.

Speaking of a caller function, we can create the plotNaurt function now

func plotNaurt(response NaurtResponse) (string, error) {

	traces := []grob.Trace{}

	var layout *grob.Layout

	if response.BestMatch != nil {
		extractNaurtInner(response.BestMatch, &traces, &layout, true)
	}

	if response.AdditionalMatches != nil {
		for _, data := range *response.AdditionalMatches {
			extractNaurtInner(&data, &traces, &layout, false)
		}

	}

	if layout == nil {
		return "", errors.New("no best match found")
	}

	fig := &grob.Fig{
		Data:   traces,
		Layout: layout,
	}

	jsonFig, err := json.Marshal(fig)
	if err != nil {
		return "", err
	}

	return string(jsonFig), nil

}

Make sure you import "encoding/json" and "errors" here.

Essentially, we use the Plotly package to convert Naurt into a map. We’ll actually return a JSON as a string from this function, since in order to serve this to the user we’ll place it inside a template.

Templating the Response

We’re going to use templates the serve the response to the user. Let’s make a folder and template file for this purpose

$ mkdir templates
$ touch templates/index.html

Inside index.html place the following template code

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <title>Plotly Map</title>
  <script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
</head>
<style>
.container {
    width: 100%;
    height: 100vh;
}
</style>
<body>
  <div class="container" id="mapDiv"></div>
  <script>
    var data = JSON.parse({{.Data}});

    console.log(data)

    Plotly.newPlot("mapDiv", data.data, data.layout);
  </script>
</body>
</html>

The basic idea is we’re going to template in a JSON, which will be parsed into an object and then contain the necessary data for creating the Plotly plot. Unfortunately, the Plotly package in Go doesn’t have some kind of function which converts the plot to HTML in memory (it does have one that converts it to a file, and while I didn’t benchmark I assume this is faster than then reading that file off the server disc and sending it).

Back in main.go we can add this near the top of the file, outside any function. Make sure to import html/template (not text/template which is very similar but won’t work!)

var tmpl *template.Template

tmpl will store the template which we can use over and over again. We’ll need to initialise it somewhere and the main function is the most natural

func main() {

	initialiseApiKey()

	tmpl = template.Must(template.ParseFiles("templates/index.html"))

	http.HandleFunc("/", handler)

	fmt.Println("Starting the server!")
	if err := http.ListenAndServe(":8080", nil); err != nil {
		fmt.Println(err)
	}
}

We’ll create a small helper type which will pass the data to the template

type PageData struct {
	Data string
}

Now all we have to do is update the handler function to actually use the template

func handler(w http.ResponseWriter, r *http.Request) {

	query := r.URL.Query()

	address := query.Get("address")

	var latitude *float64
	if query.Has("latitude") {
		tmp, err := strconv.ParseFloat(query.Get("latitude"), 64)
		if err != nil {
			fmt.Fprintf(w, err.Error())
			return
		}
		latitude = &tmp
	}

	var longitude *float64
	if query.Has("longitude") {
		tmp, err := strconv.ParseFloat(query.Get("longitude"), 64)
		if err != nil {
			fmt.Fprintf(w, err.Error())
			return
		}
		longitude = &tmp
	}

	resp, err := makeNaurtRequest(address, latitude, longitude)
	if err != nil {
		fmt.Fprintf(w, err.Error())
		return
	}

	mapJson, err := plotNaurt(resp)
	if err != nil {
		fmt.Fprintf(w, err.Error())
		return
	}

	data := PageData{Data: mapJson}

	e := tmpl.Execute(w, data)
	if e != nil {
		fmt.Fprintf(w, e.Error())
	}
}

And we’re done! If we compile and run this code again, then we’ll get a map when we go to for example http://localhost:8080/?address=The Grand Hotel, Brighton

‍