Virtualisation, the greatest tool you don’t [currently] use

Originally posted on xyHt:

This post is dedicated to a good friend, Chris Johnson who, when we worked together a few years back, introduced me to this cool new word….VIRTUALISATION. You see, Chris is an IT guru (see: Enterprise Systems Architect), a friend that every GIS guru should have so that there is someone there to pick up the pieces when you are throwing it out of the window. In the 5 or 6 years we worked together, I threw lots….mostly due to having to deal with extremely large data and very slow computation.

At this point, I need to explain that I was using a HP Z600 workstation with tonnes of Xeon processors, a gaming graphics card and 64GB+ of RAM (we had already identified Esri’s contiguous RAM requirements). The machine SHOULD have been a solution for everything but as soon as I got some survey data in, usually 1TB of xyz data, I’d go to chop it up or interpolate to raster and be waiting a week…A WEEK! When you are trying to justify your worth at a job and costing the company approximately £4500 a week just for converting some geospatial data it made for some interesting annual reviews.


Server serves “virtual computers”

“It’s not me it’s the hardware”

So, the day came that Chris and I had a chat, it’s always the IT guys fault, right?…Well, Chris asked why I was running the processes on my desktop when the company had a Windows 2012 server with hundreds of cores and RAM. He might be able to virtualise the software so that the server could work as the hardware to churn through the data. Now, all these years on, it doesn’t seem so crazy, afterall almost all the top survey companies like Trimble, Hexagon and TopCon have a cloud based engine. What is this cloud based engine? Yup, you guessed it, it is essentially a server or bank of servers that just “churn” through data. Processing the data in this manner is typically called “Cloud Processing”

Let’s have a think about what a modern workplace server is, a few decades ago they were systems in place to serve as an internal email and storage facility. Recently, company servers have developed substantially, just looking at the Windows 2016 Server operating system makes your mind boggle, it is more computer OS (Operating System) than server OS. The GUI (Graphical User Interface) opens the same as the desktop counterpart and all the applications reside under the Windows icon, it is scarily similar, in fact I have been told that the server and desktop OS share most of their infrastructure.

The Server 2016 Interface


The magic of desktop virtualisation…

If you’ve got this far, I am assuming you get the idea, using all those cores and RAM makes it go faster, great for processing. This is only part of the story, imagine being able to log into any computer, anywhere and seeing your desktop, exactly as you would on your home machine, all your software is there and works just as quick (if not quicker). Recently I was being shown the familiar windows desktop with Microsoft Office from an Apple iPhone, I kid you not. Where all the “grunt” is done on the server, the GUI only needs to send and receive the small changes visually which occur from the interactions. This means that you can quite easily perform those mammoth ArcGIS, QGIS or Pix4D jobs from your phone (providing there is a signal).

Why aren’t you using it right now? Simply put, cost. On a small scale, although the price of servers and rack space have come down a LOT in price, there is still a price to be paid, unless you are processing A LOT of data it might just be better to wait those extra couple of days for the odd job here or there.

Many servers make light work

But don’t despair yet, a few years ago cloud processing was a thing of dreams and now the likes of Trimble, Leica, Topcon et al, provide it as part of their offering. Although many large businesses may run a basic virtualised environment, a lot more of the medium and smaller sized businesses are virtualising their systems due to how cost effective it can be on licenses, downtime and time lost to processing, how long will it be until virtual desktops with GIS processing software like QGIS, Meshlab, ArccGIS & Pix4D are available to rent? I don’t think it will be long!

White Paper on Virtualisation of ArcGIS Pro from 2015:

Desktop Virtualisation (using Server 2016):

More on virtualisation from providers, VMWare:


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.