Gamma Blog: Serverless in GIS

Serverless in GIS

As GIS systems become more widely used, there is increasing demand for infrastructure to support them. The obvious way to extend processing capacity for GIS application is by scaling horizontally and simply adding more machines. Serverless technologies now offer a more beneficial alternative.

Historically, to host any GIS or IT system, there are a number of requirements:

    1. server – a machine to run the system
    2. redundancy – two servers to allow for one falling over
    3. multi-location – two servers in two locations for geographic redundancy
    4. scaling on demand – turn off servers during quiet times
    5. deployment – synchronised rollout of new data/code to all servers.

Now this can all be accomplished using serverless technologies. There are many options available from the big players and more. Google has ‘Cloud Functions’, Microsoft has ‘Azure Functions’ and Amazon has ‘AWS Lambda’.

What is serverless?

Serverless is where you define a piece of code and your cloud provider will execute it for you and return the result. In essence, they are still running servers under the hood but they take all the responsibility of allocation and managing the machine resources. We chose to use AWS Lambda.

How does serverless improve GIS?

    1. Simplicity for development – write some code or functions and upload to your cloud provider and they will execute it whenever requested. AWS Lambda supports .Net, Node.js, Java, Python and Go.
    2. No need to boot up a machine – you don’t need to build or boot a machine as it is all handled by Amazon Lambda.
    3. Instant scaling – each call to the Lambda function will execute in parallel so the user gets out-of-the-box scaling.
    4. Single item deployment – to roll out new code, simply upload your latest code to one location and your deployment is complete.

    1. Pay for what you use – the pricing model is per execution so there’s no need to pay for a server that is running 24/7.

Serverless tile server

Autoaddress created a tile server using AWS Lambda functions to display the red dot layer for the location of each building on the Eircode finder website. By using AWS Lambda functions, we were able to design a system that involved a minimal amount of moving parts which still met all the requirements and was even cost-effective.
The requirements were:

    1. Each building that has an Eircode appears as a red dot.
    2. Draw any tile that contains a red dot.
    3. Overlay on OSI maps with building outlines.
    4. Capacity to serve 250,000+ tiles per day.
    5. 100% uptime required.

We accomplished this by writing an AWS Lambda function to create the .PNG files as tiles that match the OSI tiles’ size and zoom levels and for efficiency we used lazy processing, which meant we only created each tile once and only when required.

Lazy processing

With this process, we only create tiles the first time they are requested and then save them to the tile data store. Any subsequent request will return the tile directly to the user so it will not need to be generated again.
The AWS Lambda tile creator returns the generated image back to the user and sends the image to the AWS Lambda tile saver function in parallel.

First time access:

 

Each subsequent access:

 

Gamma Blog: Serverless in GIS - lazy processing 2

Serverless tile server

Autoaddress has been using the serverless tile server displaying over 1 million tiles per week for the Eircode finder website since early 2017. As expected, the tile creating peaked when the system went live and has reduced continuously as more and more tiles get saved to the tile data store. The system has proved to be highly cost-effective and maintained a 100% uptime over the last year.
Serverless is a step forward in generating technology and suits GIS very well. Not only does it modernise the systems, but it even gives additional benefits and cost savings along the way.

@ 2018 Gamma.ie by Caoimhín Blake

About Autoaddress

Autoaddress, Ireland’s leading solution for managing addresses and Eircodes, is a set of simple APIs and tools that enable you to quickly capture accurate addresses. You can learn more, request the developer key or book a demo here.