Tag: OSTN02

Add OSTN15 to QGIS 2.16

dragons8mycat4 Comments

As you may be aware, the United Kingdom has a new transformation model that is OSTN15…..But why? What does it mean to the geospatial community?

Without being too nerdy, tectonic plate movement means that the “model” surface (the geoid) is slowly moving from best fit for the coordinate system. It has been 13yrs since Ordnance Survey implemented OSTN02 so the shift since then is enormous…..a whole 1cm and vertically it is 2.5cm. See this article here from Ordnance Survey.

The whole story is that sensors and our ability to calculate our positi0n relative to both the mathematical models and our relative position to those is constantly evolving too. So, just as OSTN02 revolutionised the accuracy of projecting GPS (WGS84) coordinates using a grid transformation (250 points over the 7 parameters used until 2002), OSTN15 both uses the OS Net of 250 points but has also been improved further with 12 zero order stations with accuracy of 2mm horizontal and 6mm vertical.

So how will this change the way you use your GIS?

If you are already using OSTN02 for your transformations between EPSG 27700 and EPSG 4326 – then you will only see a 5cm improvement over a 1m area at best and this is based on the worst places in the UK, on average you will only see a 2cm improvement anywhere in the UK. To put this into context, when you are zoomed in to an A3 map to about 1:100, you are talking about a few pixels on the screen….it won’t be groundbreaking [at the moment].

Currently, as this goes to press, the OSTN15 transformation has only been available for a few weeks and it is still being tested on different software to ensure it works, I am told that ESRI UK have been testing it with their software as this is being written.

As with OSTN02, I’ve created a fix for QGIS and OSTN15, I will describe how to implement this further in this.

It’s all about the Proj

Proj (Proj.4) is a cartographic library which is based on the work of Gerald Evenden of USGS back circa 1980. Over time it has evolved to consume grid transformations and is used by GRASS GIS, MapServer, PostGIS, Thuban, OGDI, Mapnik, TopoCad,GDAL/OGR as well as QGIS.

There are many ways to use proj, without a GIS you can use it through a command line by defining parameters. QGIS uses the proj library by accessing a spatialite database called srs.db. This is held at .appsqgisresourcessrs.db in Windows and Linux.

The proj spatialite database is a relational database which, when analysed, holds tables for coordinate systems, epsg codes & transformations. What is really clever is that it recognises direction of transformation.

Why is direction important?

Most coordinate transformations go from the projected coordinate system to the geographic coordinate system, for example epsg 4277 to epsg 4326, OSTN15 bucks the trend and is the reverse direction, from 4326 to 4277.

As I found when I first tested OSTN15 with QGIS, I was getting a uniform 200m shift in the data which was being translated and I was really confused. After talking with the gridfile creator, I discovered that the file was created from ETRS89 to OSGB36, therefore the 200m shift I was getting.

QGIS is awesome, you’ve probably overlooked just how clever it is and so did I. Next time you run a transformation, or when you try this one, you may notice that there are 2 fields noted in the columns SRC (source) and DST (destination)…and this is a godsend for solving this issue, as QGIS can read the coordinate in both directions.

transformation-in-qgis

Show us the magic

So, I talked with Ordnance Survey and found that OSTN15 has been given the epsg of 7709 and created a new record with the srs.db which is distributed with Windows, Linux & Mac releases. To utilise this, all you need to do is to download the OSTN15 file from Ordnance Survey (here) and then place the OSTN15_NTv2.gsb file in the shared projections folder .shareprojOSTN15_NTv2.gsb this has been found to be correct in Mac and Windows (there should be similar in Linux). You know it is the right folder as there should be other .gsb files in there!

qgis_folder_location

You can download the updated srs.db from here, this should be placed in the resources folder which can be found at  .appsqgisresourcessrs.db – I highly recommend changing the name of the srs.db file in this folder to something like srs.db.old before adding the new version, just in case it doesn’t work for your particular set up BUT it has been checked on Mac and Windows distributions of QGIS from version 2.12 through to QGIS 2.17.

Enjoy

Dragons8mycat

 

Many thanks to Ordnance Survey for their help

Further reading about the model for Great Britain and OSTN15, I recommend this paper: A guide to Coordinate systems in Great Britain

GIS Tips – Getting OSTN02 to work with QGIS 2.2

dragons8mycatLeave a comment

If you follow me on Twitter, you will know how frustrated I have been lately with QGIS and getting the Ordnance Survey transformation to work fully. With QGIS 2.2 came a lot of great changes, the best for me was the ability to add custom transformations, this gave me the ability to apply 7 parameter transformations to my data which would get within 4m accuracy (2m on average).

Recently Sourcepole released a blog which highlighted how grid transformations could be added directly to the core SRS database in QGIS. I’ve adapted what is shown on here to work with the OSTN02 NTv2 file.

First a small disclaimer – This is the method which I used on a Windows 8.1 machine running QGIS 2.2 on an OSGEO4W 64bit install, if you are a developer and have a better method, pleeeeeaaaasssse provide it, otherwise I provide this as an easy fix until this is made part of the next QGIS release.

Required:

Steps:

1. Unzip the OSTN02 zipfile and save the OSTN02_NTv2.gsb file to the ./share/proj folder of your QGIS install (for the OSGEO4W, this is C:/OSGEO4W/Share/Proj/ )

Image

2. Open the SQLite browser you installed and click “File” and then “Open Database”. Navigate to .\apps\qgis\resources\ and open the srs.db file.

Image

3. Once open, select the “browse data” tab and select the tbl_datum_transformation table.

Image

4. Scroll to the bottom of the table, the cells can be changed by double clicking on each cell in turn and changing the value, then selecting the bottom right “save changes” option. Once all changes have been made, save the table using the save option at the top left of the GUI.

Image

For my version, I just changed the values in the last record (line 775). The values I used were as follows;

epsg nr: 27700

coord op code: 100012

source crs code: 4277

target crs code: 4326

coord op method: 9615

p1: OSTN02_NTv2.gsb

p2: LEAVE BLANK

p3: LEAVE BLANK

p4: LEAVE BLANK

p5: LEAVE BLANK

p6: LEAVE BLANK

p7: LEAVE BLANK

Remarks: Put what you want in here, I put the Ordnance Survey disclaimer text to ensure copyright was adhered to.

Scope: As above, put what you want

Preferred: 1

depreciated: 0

area of use code: 1264

Image

5. Goto line 210 (should be the Helmert toWGS84 transform) and change the preferred cell to 0 and the depreciated cell to 1.

6. Save table and close.

7. Open QGIS and go to “settings” and then options. Remove any default datum transformations and then ensure the “ask for datum transformation” option is selected.

Image

Add a WGS84 data and then an OSG36 data into the same frame, when the OSGB36 is added you should get a pop up with the option to use the OSTN02!!

Image

So far I have found this to be reliable and and levels of shift are almost invisible, inline with the 0.001m claimed by Ordnance Survey.

Now that you have changed the srs.db you won’t be too interested in downloading the one here to replace yours to save you time.

If you need 2 files to test, there are 2 shapefiles (1 in WGS84 & 1 in OSG36) available here, they are both the same data but in different coordinate systems and should perfectly align, if they don’t, then there is a projection shift problem!

Nick D

GIS Tips: Adding OSTN02 to ArcGIS Explorer (build 2500)

dragons8mycatLeave a comment

Adding OSTN02 to ArcGIS Explorer Build 2500

This is a quick walkthrough of how to apply OSTN02 to ArcGIS Explorer build 2500, it assumes a Windows 7 install, though this fix will work with XP.

First download & install the Projection System expansion pack: http://resources.arcgis.com/en/communities/arcgis-explorer-desktop/01s000000017000000.htm#ESRI_SECTION1_5F289729636840309B87C664AEEF4249

Then navigate to the ArcGIS Explorer program folder, this can be found at: C:\Program Files (x86)\Explorer (x86)\

1

Navigate to the projection engine (pedata) folder:

2

Add a new folder and call it “united kingdom”:

 

3Next you need to download the OSTN02 transformation from Ordnance Survey, the zipfile can be found here: http://www.ordnancesurvey.co.uk/business-and-government/help-and-support/navigation-technology/os-net/ostn02-ntv2-format.html

Select the “Download OSTN02 NTV2 data” option:

4
Within the zipfile you will see 3 files, the only file you need is the OSTN02_NTV2.gsb file.  Copy this into the “united kingdom” folder you created.

5

The transformation should now work!! Start up ArcGIS Explorer desktop and load in a OSGB36 dataset, go to the map properties and select “2D Coordinate System”, when you click on “Transformation” next to OSGB36, you should see this:

6

GIS Tips – Using OSTN02 with QGIS

dragons8mycat1 Comment

Not often I get requests….well, not often I get requests which lay down the guantlet. In my previous post “GIS Tips – Don’t trust your GIS, have you heard of OSTN02?” I am quite flippant when mention how to add the OSTN02 to the QGIS system, then I got a couple of replies to my blog pointing out that it really isn’t that easy….

I’m not one to run away way the going gets tough, so here goes….

Adding the OSTN02 transformation to QGIS

First download the OSTN02 transformation from the Ordnance Survey website: here

Then extract the files to a folder you will easily be able to find later. Essentially all you REALLY need is the OSTN02_NTv2.gsb file – The GSB format is also NTV2 format (I believe) and is a grid of binary transformations.

Okay, here is the tricky bit, please only attempt this if you are on QGIS 1.8 and above….

Navigate to C:\Program Files\Quantum GIS Lisboa\share\proj\ 

Copy the OSTN02 gsb file into this folder

Open QGIS and add the transformation tool add-in. Without this addi-in all this would be impossible!

Image

Obviously set WGS84 and British National Grid as your 2 coordinate systems and away you go!!
I tested this (as you can see above) and it works great!! Some might say it is even easier than the ESRI install!!

Nick D

GIS Tips – Don't trust your GIS, have you heard of OSTN02?

dragons8mycat3 Comments

You wouldn’t believe the number of GIS consultants. managers and technicians I have met who haven’t heard of OSTN02, what is more worrying some of them work in prominent roles and make some big decisions.

Okay, so if you don’t live in the United Kingdom or work with UK spatial data, then you can switch over now, though if you do live in the UK and work with UK spatial data in any form and you haven’t got a clue what I am rattling on about, read on.

What is OSTN02?

Ordnance Survey Transformation Network 02, OSTN02, is a geodetic transformation for  Ordnance Survey British National Grid….in plain English, it is a transformation for working between WGS84, ETRS89 & OSGB36.

Coordinate systems such as WGS84 & ETRS89 are Geographic whereas OSGB36 is Projected (image care of ESRI)

A quick catch up on British National Grid (It will help later)

The grid is based on the Ordnance Survey Great Britain 1936, OSGB36 datum (based on the Airy 1830 ellipsoid), and was introduced after the retriangulation of 1936–1962.

The Airy ellipsoid is a regional best fit for Britain; more modern mapping tends to use the GRS80 ellipsoid used by the GPS (the Airy ellipsoid assumes the Earth to be about 1 km smaller in diameter than the GRS80 ellipsoid, and to be slightly less flattened).

Over the Airy ellipsoid a straight line grid, the National Grid, is placed with a new false origin (to eliminate negative numbers), creating a 700 km by 1300 km grid. This false origin is located south-west of the Isles of Scilly. The distortion created between the OS grid and the  projection is countered by a scale factor in the longitude to create two lines of longitude with zero distortion rather than one.

Random fact: OSGB 36 was used by UKHO for Admiralty Charts in UK waters until 2000, after this WGS84 was used!

Why you can’t trust your GIS

The most common geodetic transformation from WGS84 to a projected coordinate system (such as OSGB36) converting between  is called the Helmert datum transformation, this is a 7 parameter transformation which results in a typical 7m error from true.

It is at this point that you need to start up your ArcGIS or QGIS and remind yourself about the transformations which you are using. If you are using ArcGIS 3.2 – 10, you will most probably be using one of these Helmert transformations:

Table from ESRI geoXchange

If this isn’t enough evidence that you need to move to the OSTN02 transformation, then look at the map below by Steve Baker which shows the difference between OSTN02 and the ESRI recommended WGS84_to_OSGB36_Petroleum:

Convinced? So how does this OSTN02 work?

Rather than use the Helmert 7 parameter transformation, the OSTN02 uses hundreds of reference stations (OS Net) throughout the UK:

https://i0.wp.com/www.ordnancesurvey.co.uk/oswebsite/images/products/OSNetWebSiteMap_fullsize_pluscolours.jpg

OS Net stations 2012 thanks to Ordnance Survey

 Using the OS Net the transformation models the detailed distortions in the 1936–1962 retriangulation of OSG36, and achieves backwards compatibility in grid coordinates to the sub-metre accuracy.

For more detailed information, the Ordnance Survey has a wealth of free resources covering this: Read more here

Okay, very clever, but how can I use it?

Although the OSTN02 doesn’t ship with ArcGIS 3.1 – 10.1 (at time of printing), Ordnance Survey have worked very closely with ESRI UK to enable the transformation to be used, it can be downloaded free of charge from here: OSTN02 add-in for ArcGIS

To utilise this transformation method simply download and paste the OSTN02_NTv2.gsb into a folder called ‘C:Program FilesArcGISDesktop10.1pedatantv2uk’. You will need to create the folder called ‘uk’ which must be in lower case. Next time you restart ArcGIS 10.1 you will be able to use this transformation from the Geographic Coordinate Systems Transformations dialog box

This transformation is also available for QGIS through Ordnance Survey and can be downloaded from here: OSTN02 in NTV2 format

Around here I should post some instruction about how to install it, problem is that I haven’t been entirely successful in getting it to work, hopefully the amazing QGIS community will (as usual) point out some stupid spelling mistake I am making and I will be able to update this post shortly to provide a little user guide (it seems easy enough!!).

Nick D

GIS Tips – Don’t trust your GIS, have you heard of OSTN02?

dragons8mycat2 Comments

You wouldn’t believe the number of GIS consultants. managers and technicians I have met who haven’t heard of OSTN02, what is more worrying some of them work in prominent roles and make some big decisions.

Okay, so if you don’t live in the United Kingdom or work with UK spatial data, then you can switch over now, though if you do live in the UK and work with UK spatial data in any form and you haven’t got a clue what I am rattling on about, read on.

What is OSTN02?

Ordnance Survey Transformation Network 02, OSTN02, is a geodetic transformation for  Ordnance Survey British National Grid….in plain English, it is a transformation for working between WGS84, ETRS89 & OSGB36.

Coordinate systems such as WGS84 & ETRS89 are Geographic whereas OSGB36 is Projected (image care of ESRI)

A quick catch up on British National Grid (It will help later)

The grid is based on the Ordnance Survey Great Britain 1936, OSGB36 datum (based on the Airy 1830 ellipsoid), and was introduced after the retriangulation of 1936–1962.

The Airy ellipsoid is a regional best fit for Britain; more modern mapping tends to use the GRS80 ellipsoid used by the GPS (the Airy ellipsoid assumes the Earth to be about 1 km smaller in diameter than the GRS80 ellipsoid, and to be slightly less flattened).

Over the Airy ellipsoid a straight line grid, the National Grid, is placed with a new false origin (to eliminate negative numbers), creating a 700 km by 1300 km grid. This false origin is located south-west of the Isles of Scilly. The distortion created between the OS grid and the  projection is countered by a scale factor in the longitude to create two lines of longitude with zero distortion rather than one.

Random fact: OSGB 36 was used by UKHO for Admiralty Charts in UK waters until 2000, after this WGS84 was used!

Why you can’t trust your GIS

The most common geodetic transformation from WGS84 to a projected coordinate system (such as OSGB36) converting between  is called the Helmert datum transformation, this is a 7 parameter transformation which results in a typical 7m error from true.

It is at this point that you need to start up your ArcGIS or QGIS and remind yourself about the transformations which you are using. If you are using ArcGIS 3.2 – 10, you will most probably be using one of these Helmert transformations:

Table from ESRI geoXchange

If this isn’t enough evidence that you need to move to the OSTN02 transformation, then look at the map below by Steve Baker which shows the difference between OSTN02 and the ESRI recommended WGS84_to_OSGB36_Petroleum:

Convinced? So how does this OSTN02 work?

Rather than use the Helmert 7 parameter transformation, the OSTN02 uses hundreds of reference stations (OS Net) throughout the UK:

https://i0.wp.com/www.ordnancesurvey.co.uk/oswebsite/images/products/OSNetWebSiteMap_fullsize_pluscolours.jpg

OS Net stations 2012 thanks to Ordnance Survey

 Using the OS Net the transformation models the detailed distortions in the 1936–1962 retriangulation of OSG36, and achieves backwards compatibility in grid coordinates to the sub-metre accuracy.

For more detailed information, the Ordnance Survey has a wealth of free resources covering this: Read more here

Okay, very clever, but how can I use it?

Although the OSTN02 doesn’t ship with ArcGIS 3.1 – 10.1 (at time of printing), Ordnance Survey have worked very closely with ESRI UK to enable the transformation to be used, it can be downloaded free of charge from here: OSTN02 add-in for ArcGIS

To utilise this transformation method simply download and paste the OSTN02_NTv2.gsb into a folder called ‘C:\Program Files\ArcGIS\Desktop10.1\pedata\ntv2\uk’. You will need to create the folder called ‘uk’ which must be in lower case. Next time you restart ArcGIS 10.1 you will be able to use this transformation from the Geographic Coordinate Systems Transformations dialog box

This transformation is also available for QGIS through Ordnance Survey and can be downloaded from here: OSTN02 in NTV2 format

Around here I should post some instruction about how to install it, problem is that I haven’t been entirely successful in getting it to work, hopefully the amazing QGIS community will (as usual) point out some stupid spelling mistake I am making and I will be able to update this post shortly to provide a little user guide (it seems easy enough!!).

Nick D