Ocean Business 2017

Originally posted on xyHt 17/4/2017

Last week, April 4th – 6th, one of the largest gatherings for the offshore industry took place, Ocean Business, in Southampton, UK. The bi-annual event is a large deal for the maritime industry. When I say large, I actually mean huge; there are over 350 exhibitors from 26 countries and 166 training and demonstration workshops in just those three days!

What is amazing about Ocean Business is that the whole maritime industry is represented. Of course there are the surveyors, the geologists, and the equipment hire companies, but also there are representatives from the GIS, CAD and engineering sectors all sitting side by side.

At the last event talk was about the latest Reson multibeam device. Whereas before conversation would have been around capturing scour and wreck details, this year was talk of analysing pipelines for cracks or whether bolts were where they should be. It was hard at times to remember that I was at an offshore event as the technology has developed so quickly.

Of course, in two years, technology has shrunk. With the onshore industry, this is not overly exciting but offshore this is BIG news. Take, for example, the Planet Ocean micro AUV, developed with the National Oceanographic Centrre. This micro-UAV is only 50cm long and shaped like the traditional “torpedo”. Being fully autonomous allows it to capture data in ways never previously considered.

Big improvements to Doppler Velocity Log (DVL) technology was demonstrated by most of the equipment exhibitors like Sonardyne, Nortek and Kongsberg. If you don’t know what a DVL is, it how you get location when you are underwater and have no GPS/GNSS. The systems on show were capable of capturing the location at better than 0.01m at depths up to 6000m. Just think that over and relate that to your super cool iPhone!

The data providers and geospatial companies who were present were murmering about BIM and how it is going to start to become an issue. How does BIM integrate with the information captured and provided offshore? Can the maritime industry build a common spatial data infrastructure?

One company I met that were really making strides on this was OceanWise. Dr Mike Osborne, a respected hydrographer, has taken it upon himself to get the maritime industry thinking about these issues and working towards a solution. OceanWise provides nautical geospatial data but also trains the offshore and onshore industries in the use of GIS for nautical charting, including maritime standards. Mike  has set up maritime spatial data infrastructure groups across the world with 33 countries already involved, something which is unheard of in the onshore industries. Mike has agreed to be interviewed by xyHt, so look out for that in a later issue!

My final comment is about how much Lego was on display. I mean, do these guys have nothing better to do with their time? As much as there was UAVs, transponders, ROVs and new vessels, there were Lego survey vessels and tech remodelled in Lego. Maybe it is because I wasn’t allowed to take one home that I am so curious about it.



Click on an image below to see the larger version


Explain Georeferencing To Me as If I Were a Five-Year-Old

Blog post copied from Adrian Welsh on GeoNet 30/11/2016 – Too good not to share!!

Explain Georeferencing To Me as If I Were a Five-Year-Old


I really liked how Denzel Washington used the phrase “explain this to me as if I were a xxx-year-old” in the movie Philadelphia (1993).

Reference: Philadelphia. Philadelphia, PA: Jonathan Demme, 1993. film.

So, I will take it one step further and attempt to explain the concept of georeferencing to an actual five-year-old.


Five-year-old engineer says, “I have this PDF of a site plan. I want to put this on a map and have it line up properly.”

Here is my map.

We need to zoom in a little bit closer.

A little bit more.

Open Street Map 1:5,000

Almost there. Zoom in some more so that our site plan will fit better.

Open Street Map 1:1,050

Much better. Now, we need to shrink the site plan to a more usable size. Currently, it’s larger than our map.

Let’s make it a little bit smaller.

Perfect. Now we need to place the site plan on our zoomed in map and adjust it to fit by rotating it and resizing it.

Great! Now, after some quality control of adjustments and transformations, we can rectify this image and call it georeferenced!

OSM 1:1,050 with Image

We can make the georeferenced image transparent to where we can see the basemap behind it.

OSM 1:1,050 with Image, Transparency 50%

Finally, we can add existing linework and other GIS files to give the image a more solid reference.

OSM 1:1,050 with Image, Transparency 50% and Linework

Many thanks to Adrian Welsh for letting me share this!


Vertical transformations in GIS

You know those moments where you are sat in a pub, the office or at a friends house and you say something, then suddenly wonder why you haven’t thought of it before?….I had that very issue in the Brew Dog pub near Spitalfields in London, UK a couple of weeks ago. You see, I was discussing the use of web mapping in 3D and writing a specification for the data being hosted and then it came out….

“I wonder if 3D GIS systems use 3D transformations”…...

Before I even finished the sentence I was starting to get palpitations at the thought of throwing all my hard work over the last few months away. Okay, so maybe that is a bit of an exaggeration. I do take into consideration the vertical transformation when working between data and am a bit of a tide datum nerd, though the question was now out there….

3D GIS…a quick reminder

First, let me address the “3D GIS” elephant in the room. Is there such a thing? My current stance is yes. Having used many different GIS over the last few years, I would say that many would tell you that they are, but they would almost all be wrong. Take for example CesiumJS, it is a fantastic 3D webgl viewer but can you edit the data on screen? Can you transform data between coordinate systems? (The answer you are looking for is no) Other systems which I have discounted are QGIS, MapInfo, CADCorp, Google Earth, ArcScene, GRASS, OpenGIS and also AutoCAD Map¹, purely due to them not having the capability of showing multiple coordinate system 3D data on a globe. Yes, I understand that you can pre-process the data so that it is all in the same coordinate system and then process the datum so that it is all consistent, but NOT on the fly.

There is only one “3D GIS” at the moment, the much overlooked ESRI ArcGIS Pro.


Compared to the 3D capabilities of the other GIS, asking it to handle vertical transformations on the globe as well as the most up to date horizontal transformations would be a bit of an ask, but you know what?…..It is pretty good…..

Disclaimer time, I am currently running the Beta version of ArcGIS Pro 1.4, though I am told it shouldn’t matter, also I have been talking with a few of the developers on functionality but this is a 3D GIS which is out on the shelves now and works. For the OCD spatial nerd, it ticks the boxes, I can use the provided international transformations or I can create my own, though more importantly, I can work with my survey (lidar) data alongside my topographic data, which is alongside my hydrographic data- so I can easily QA the data and share the results quickly.

For the n00bs, why do you need vertical transformations?

Simply put, a horizontal datum is a reference system for specifying positions on the Earth’s surface, so that the coordinate system is of uniform measure, likewise, a vertical datum is a reference to the height of the surface. All simple at the moment….a datum is based on an ellipsoid/spheroid (these geo people can’t decide what shape round they want to call it, so interchange all the time) Each ellipsoid is the best approximation of the Earths surface at any point on the Earth, you may have seen “Clarke 1866, GRS80 or WGS84” in the coordinate information of your data


So you can start to see, just transforming your 3D data from one coordinate system to another isn’t quite so simple as the 2D Helmert you’ve been using. Relax, I’m not going to now go into a geoids and equipotential surfaces (unless you want me to) I am merely pointing out that we need vertical transformations in out lives, otherwise, when you transform your beautiful 3D village model from WGS84 to British National Grid, you will be wondering why it is floating in the air.

This might not matter much to the casual 3D GIS dabbler, as most of the work may be small and the effects won’t be large enough to be noticeable, though when it gets to city size, you will certainly start seeing issues with the buildings at the periphery not sitting right (when you edit the centre to sit right).

In summary

Since joining Garsdale Design in January, I have been doing a LOT of 3D GIS and putting my geodetic knowledge to ensuring things are correct. In the last 6 months I have been having to use ArcGIS Pro more and more, partly due to functionality and partly due to data formats which clients use. My opinion is that this software is becoming the one stop shop for 3D GIS and it is capable of supporting the whole project lifecycle of the project….it could turn out to be the BIM solution that everyone has overlooked. More importantly though, is that ESRI have done their homework and are supporting vertical datum transformations….okay, there are a few missing here and there BUT the list is growing and they are putting these vertical transformations into the geoprocessing tools of ArcGIS Desktop too.

Screenshot of (a small portion of) the vertical transformation list

If you know of another 3D GIS, please let me know, as I’d be eager to try it. Please feel free to either contact me direct or through comment below.

QGIS – What do you do when you move your .qgs file?

What do you do when you move your file location in QGIS and lose all your links? Maybe try this….

So, the situation occurred yesterday where I was giving a workshop and sent out a load of QGIS styles, layer definition files and also a project file (.qgs)….Smugly, I told everyone to open the project file, then realised, as hands raised across the room that QGIS doesn’t work with relative paths and it also doesn’t do a “map package”. Working with so many different GIS, it’s hard to keep track of which ones do different things but I really should have remembered this one.

Surprisingly, the solution to repairing all the links and getting it all up and running is relatively easy if you are working with disconnected databases or vector files (shapefiles etc). Just make sure you have a text editor and away you go….


First of all, open the rogue .qgs file in your text editor, in the example above, I am using sublime text editor but during the workshop I found Windows notepad was just as capable. Upon opening, you will see that the project file is just a standard xml file with references to several processes.

Use your “Find” option in the text editor to find one of the <datasource> tags (as shown above)

It is simply a case of then changing the folders within that datasource tag to locate the correct location (most people store their data in a single location).


As you can see above, I want the project to read all the data from C:OS Southampton rather than the G:Work_Admin_Backup_Nov15GIS Core DataOS Southampton location, so using the REPLACE function (sometimes called the find/replace in some text editors) we can simply change ALL the locations in one go.

Pretty easy huh? A lot easier that using the interface which is provided by QGIS for updating each file link individually, after all, most times we just change folders, we don’t scatter our data around a drive location.

I am sure that this sort of functionality (changing the folder to reference all the links) could be done in bash or as an extra option within QGIS, if you know how, I look forward to hearing from you!


Mapmaking – "Can I have a map?"

Originally posted on xyHt.com

“Can I have a map?”

How often do you get someone ask “can I have a map”?….I have nothing against making maps, in fact I like it so much that I have made a career out of it. Not just any map either, I’ve made maps for the Nuclear, Environmental, Government, Renewable Energy, Offshore and Survey industries, to name a few, I’ve even made maps for a Prime Minister (You’re welcome Mr Blair).

My issue is the question whereby you are just asked for a map. This is not only time consuming for you, the creator but also for the poor client who you are charging 4 or 5 times over for updates to a map which should have been right the first time. This isn’t the clients’ fault, the client didn’t realise that there are multiple templates for the work or that positioning the label they wanted would take half an hour due to it causing a script in the GIS to crash. No, it is safe to assume that the client doesn’t know anything about GIS otherwise they would be doing it themselves….OR (eeek!) you could have the ultimate in intimidation that is the GIS Manager – someone who used GIS up until 10yrs ago and has forgotten how long things REALLy take *shudder*.

There are 2 was to charge people for making maps, either a fixed rate or the consultants favourite, by the hour. No matter which one you choose, it will be wrong. Charge by the hour and you spend 6hrs on 5 different versions of the map which looked fine the first time, OR you take the hit but as you make all those changes, that time you had earmarked has gone and you have to apologise to other clients who were never part of the issue. Lose, lose….


Manage Expectations

The most difficult time I had with this issue was working for local government here in the UK. One day I would be asked for a “PPL map”, the next I would be asked to map all the doctors surgeries near to elderly communities, then the next I would be asked for a map of otter holts. Every map was diverse and each required a different format, logo, information, size, paper density, north arrow…..you get the idea, no matter what, each map was unique and with little experience with making public facing maps I was taking my work home with me just to get all the work done near to deadline.

So, I created the “GIS Request Form” and I have used this in many different guises at every workplace I’ve worked and never looked back. Simply put, it is a list of basic requirements for making a map, this has sometimes be a list of 20 items but at present is a list of 7 or 8 which form the creation. If I know that the client is managing onshore assets, it is safe to assume they will require British National Grid, once things go coastal it is a completely different set of goalposts.

The basics I will always ask for are

  • Project Name
  • Time Allocated for the job
  • Presentation Size (is it for a report and to be inset or A3 folded?!)
  • Orientation of the map
  • Allow for ring-binding?
  • Description of the work (useful for context)

Other information which I ask of new clients may be:

  • The project level of accuracy
  • The stated project coordinate system
  • Preferred data provision format
  • Is this based on a previous map? If so, can a sample be porvided?
  • Is the map for internal or external use?

By having all this information at hand, you can manage the expectation a lot better and therefore saving A LOT of time. For one client, I can remember being told to run off about 25 maps for a client and I did it in the company standard A3 landscape, only for the Director to come back and tell me that the client had changed their mind and the figures had to be put into the text of the report……how do you tell a client that you are going to charge them for another 30-40hrs of work?! At least if you have it in print from the start you have a safety net!

Below is an example of the current (quick) request form…..yes, it is in the form of an email. Why? Because I ask my clients to have the message on their desktops, then if they need a map, all they have to do is fill in the details and click “send”.



First Map Perfection

Okay, so I still have to make re-iterations of the map but now I usually only make one…yes ONE. It is a personal goal of mine to ensure that a client only has to have one iteration of the map. In fact, I always tell the client that the first map is a draft and won’t be perfect as no-one has EVER made a perfect map first time (disclaimer: maybe they have, but I doubt it).

I hope this helps to smooth the workflow, feel free to comment and add your stories of client map expectations or when it has got to a crazy number of iterations!!






Add OSTN15 to QGIS 2.16

As you may be aware, the United Kingdom has a new transformation model that is OSTN15…..But why? What does it mean to the geospatial community?

Without being too nerdy, tectonic plate movement means that the “model” surface (the geoid) is slowly moving from best fit for the coordinate system. It has been 13yrs since Ordnance Survey implemented OSTN02 so the shift since then is enormous…..a whole 1cm and vertically it is 2.5cm. See this article here from Ordnance Survey.

The whole story is that sensors and our ability to calculate our positi0n relative to both the mathematical models and our relative position to those is constantly evolving too. So, just as OSTN02 revolutionised the accuracy of projecting GPS (WGS84) coordinates using a grid transformation (250 points over the 7 parameters used until 2002), OSTN15 both uses the OS Net of 250 points but has also been improved further with 12 zero order stations with accuracy of 2mm horizontal and 6mm vertical.

So how will this change the way you use your GIS?

If you are already using OSTN02 for your transformations between EPSG 27700 and EPSG 4326 – then you will only see a 5cm improvement over a 1m area at best and this is based on the worst places in the UK, on average you will only see a 2cm improvement anywhere in the UK. To put this into context, when you are zoomed in to an A3 map to about 1:100, you are talking about a few pixels on the screen….it won’t be groundbreaking [at the moment].

Currently, as this goes to press, the OSTN15 transformation has only been available for a few weeks and it is still being tested on different software to ensure it works, I am told that ESRI UK have been testing it with their software as this is being written.

As with OSTN02, I’ve created a fix for QGIS and OSTN15, I will describe how to implement this further in this.

It’s all about the Proj

Proj (Proj.4) is a cartographic library which is based on the work of Gerald Evenden of USGS back circa 1980. Over time it has evolved to consume grid transformations and is used by GRASS GIS, MapServer, PostGIS, Thuban, OGDI, Mapnik, TopoCad,GDAL/OGR as well as QGIS.

There are many ways to use proj, without a GIS you can use it through a command line by defining parameters. QGIS uses the proj library by accessing a spatialite database called srs.db. This is held at .appsqgisresourcessrs.db in Windows and Linux.

The proj spatialite database is a relational database which, when analysed, holds tables for coordinate systems, epsg codes & transformations. What is really clever is that it recognises direction of transformation.

Why is direction important?

Most coordinate transformations go from the projected coordinate system to the geographic coordinate system, for example epsg 4277 to epsg 4326, OSTN15 bucks the trend and is the reverse direction, from 4326 to 4277.

As I found when I first tested OSTN15 with QGIS, I was getting a uniform 200m shift in the data which was being translated and I was really confused. After talking with the gridfile creator, I discovered that the file was created from ETRS89 to OSGB36, therefore the 200m shift I was getting.

QGIS is awesome, you’ve probably overlooked just how clever it is and so did I. Next time you run a transformation, or when you try this one, you may notice that there are 2 fields noted in the columns SRC (source) and DST (destination)…and this is a godsend for solving this issue, as QGIS can read the coordinate in both directions.


Show us the magic

So, I talked with Ordnance Survey and found that OSTN15 has been given the epsg of 7709 and created a new record with the srs.db which is distributed with Windows, Linux & Mac releases. To utilise this, all you need to do is to download the OSTN15 file from Ordnance Survey (here) and then place the OSTN15_NTv2.gsb file in the shared projections folder .shareprojOSTN15_NTv2.gsb this has been found to be correct in Mac and Windows (there should be similar in Linux). You know it is the right folder as there should be other .gsb files in there!


You can download the updated srs.db from here, this should be placed in the resources folder which can be found at  .appsqgisresourcessrs.db – I highly recommend changing the name of the srs.db file in this folder to something like srs.db.old before adding the new version, just in case it doesn’t work for your particular set up BUT it has been checked on Mac and Windows distributions of QGIS from version 2.12 through to QGIS 2.17.




Many thanks to Ordnance Survey for their help

Further reading about the model for Great Britain and OSTN15, I recommend this paper: A guide to Coordinate systems in Great Britain

QGIS CSV & Delimited text Issues

Originally posted on xyHt Magazine 10th August 2016

Last month I was at the Maptime in Southampton (UK), helping QGIS new users how to join tables and map EU referendum maps when I came across an issue with something on QGIS I hadn’t spotted in the last *ahem* years of using it.

When you drag and drop txt, csv or other delimited files into QGIS the fields automatically get converted to text format. No, I’m not making it up and it caused a lot of embarrassment when I was giving my demonstration.


draganddrop fields
By dragging and dropping the csv file, you can see that the field type is solely “String”


This isn’t written to complain about QGIS but to notify others who are wondering why their joins aren’t working or why their interpolation can’t pick up the value field….You QGIS guys are going to say “why haven’t I raised this as an issue?”, well, firstly read Nyall Dawsons blog post on QGIS issues , secondly I tried to…..it turns out that trying to get access to submit issues has changed and even though I’ve asked for help to get access I’ve been waiting 1 month for response to request.

So…why does it happen?

If you add the file through the “add delimited file” button, none of this is an issue, this is due to the was that the software is written When the file is “dragged & dropped”, the software relies on OGR to add it as a comprehensible layer and this just renders all the fields as text (at present August 2016).

Add layer fields e2e
By adding the csv file using the add layer method, you can see the fields are brought in correctly

Why is it an issue?

If you are joining tables and aren’t aware of the issue, you drag and drop a table with a list of numerical values in, and then can’t join it to a spatial data with values in as you can’t join text to numbers. This could also cause issues with interpolation (reading of a value field) and also generation of points which need classification based on numbers.

Getting it fixed…

This is where things get a little tricky, as I don’t think it is entirely a QGIS issue and more related to the code which QGIS uses to parse the information, so until OGR update their code, it might be a bit of a wait.