What is metadata and why all the fuss?

Remember when you were a kid and when you wanted to know something, you went to this building called a “Library”? It had these things called books which were tactile paper objects which contained textual knowledge. The only problem was that you didn’t have the time to read the blurb of every book to find the one you needed. What you needed was a microfiche index, kinda like an acetate which contained all the books on the subject and a short blurb about each, also the year of publish and the author…….

Microfiche?!

Nowadays we have Google, Yahoo!, Ask Jeeves & some people are even lazy enough to use Siri, but what happens when you’re a specialist? You work on a subject where there are no books, the data you deal with is the cutting edge and then how do you share that with others so that they understand quickly and easily what the processes were to creating the opus magnum you have. I for one don’t have time to be writing a report on each data I generate.

Time to be Inspired?

For us geomagicians the answer is metadata, the so called data about our data…..I mean, just that one phrase was enough to put me off, it’s not pretty, it’s not fun but in the long run it provides you with the ability to be the master of your data and smugly turn to people, when they ask about your data, and say “Didn’t you read the metadata?”

But what is metadata and why is it useful? Why go through all the effort when no one else seems to bother? I’m going to ignore all those preaching books, the overbearing software manuals and provide my view from my years of managing Terabytes of data to some of the countries biggest companies and bodies.

Not another rant

So, the big question, why bother?

  • Adding metadata to data makes in industry standard compliant
  • Metadata provides a means to SECURE your data by adding copyrights, requirements for display, use and even legal restrictions
  • Gives you a means of tracking edits and changes so that the data history is discoverable.
  • Provides information on the date & time of creation & BY WHOM
  • Not only provides information about what the data is but also about the steps taken to create it.
  • Contains geospatial information on the geographical extents and the system the data was recorded in.
  • Metadata provides a means of providing a central point of contact so that issues and errors can be reported.
  • Metadata when created to standard, can be made into a discoverable catalogue (think wikipedia) where searches for information can be performed by those without access to the data.

Of course, there are many, many other benefits of creating metadata but depending on the format you choose, will dictate the further fields and options you have at your disposal.

One key element of the metadata is that it breaks the “geogeek” barrier. It provides a means for the non-geo people in the office to understand what data we have at our disposal and allows them to better comprehend how it can benefit them. A key example here is where you may have a whole project team accessing the data and each needs to meet their part of the project. Rather than send you endless emails or question you relentlessly for hours on end about the data, they can simply look it up and answer 92% of their questions themselves.

Even better is that the metadata can be tied to your WMS, WFS or WCS services, so that you don’t have to be available 24/7 on the other end of the phone to answer questions what is on your web maps, it can be fully described and the legal bindings on display to all.

It sounds like a lot of work….

Metadata is slowly becoming a staple requirement for the project manager, with many of the main bodies like USGS, NOAA and The Crown Estate requiring data for projects to be supplied with complete metadata. The question is surely becoming why not?

Well, the question you need to ask before embarking on the addition of gigabytes of metadata is, what format, and this isn’t an easy question. There are 4 primary formats which need to be considered, with many others which sit around the edges.

ISO 19115 – ISO 19115 and its parts defines how to describe geographical information and associated services, including contents, spatial-temporal purchases, data quality, access and rights to use.

The objective of this International Standard is to provide a clear procedure for the description of digital geographic datasets so that users will be able to determine whether the data in a holding will be of use to them and how to access the data. By establishing a common set of metadata terminology, definitions and extension procedures, this standard will promote the proper use and effective retrieval of geographic data.

FGDC (Federal Geographic Data Committee) Standard. This standard is widely used in America, was once the default ESRI format  and is very similar in construct to the ISO standard. the main differences lie in that the FGDC metadata is more catered to a data catalogue and is in a text structure. There are minor items like how originators are described and how update frequencies are defined but there there are plans to move the FGDC standard over to the ISO 19115(1) in the near future .

ESRI Metadata – The ESRI metadata format uses neither the ISO standard or the FGDC format, instead it uses a format similar to both but neglects some elements and reorganises others (a vast improvement over the 9.3 metadata!) – the information can be downloaded here

Inspire – Inspire is the european standard, it includes the ISO 19115 standard (so can be used in the UK to meet many requirements). Although it is a better standard, it is more “complete” and therefore requires more time to complete.

There are many other metadata standards, like the MEDIN standard used by the Crown Estate in the UK, this is designed more for offshore geospatial work and has fields more catered to the storage of this data. Each metadata standard should be reviewed on its benefits but in my experience, I have found that the Inspire format encompasses all the parameters and requirements to meet the ISO 19115 & also European standards.

You will hear much talk of ISO 19139 where ISO 19115 is discussed, this relates to the schema for the ISO 19115 data and that the ISO 19139 provides the implementation of xml format for the metadata. Therefore (and this can be confusing) you may have a GIS software which provides both ISO 19115 & 19139, where this is the case, the ISO 19139 will provide the data in .xml.

Meeting the standard

Most (if not all), the latest GIS software reads and allows edits of metadata in most of the most recognised standards, where they don’t the xml can be downloaded and installed or a template (xslt format) can be used.

From experience, I found the ArcGIS software easiest to view the metadata and it also had a list of metadata standards. QGIS was also easy with many tools to read/write the xml format. There are also a wealth of standalone metadata editors to create metadata to industry formats, such as CATMDEDIT or UK Location (Inspire) Metadata editor

A useful tip is that if you are using the ESRI shapefile format, the metadata is held as “yourfilename.shp.xml“. This means that you can create a single metadata with most of the generic information in, then rename the xml file with the name of the shapefile. The extents and other information are automatically updated when the file is opened, therefore you only need provide specific information about the data.

Now the big one

Okay, if you’ve got this far then you are seriously considering this fantastic resource already at your fingertips, the thing you need to equate is whether it is worth implementing. This is something that requires much consideration. The benefits are huge and once created, it is easy to maintain but the initial creation is a chore. Even more so if you have several thousand data, is it worth it though? In my experience, YES. When a client contacts me about some data, I can look it up and discuss the data in detail, even when it has been created by a colleague.

Moving forward

IF you do make the move, there are some great catalogues out there for keeping track of your metadata, one of my personal favourites is Geonetwork,  an open source solution which meets most of the primary standards discussed and here is a list of other metadata tools which may be of use.

Nick D

QGIS Tips….or should that be QTips? – QGIS on an Android Tablet

I’ve had a few people ask about setting QGIS up on Android….I can barely read phone numbers on my phone, so why you would want to use a GIS is beyond me, though, that said, when I worked for Ordnance Survey, I had a tablet with ArcGIS on and it was heavenly!

Anyhow, the short answer is that I haven’t had the chance to play with an Android build, especially on a tablet BUT I came across this great blog from Underdiver which outlines how they did it, see below

QGIS on an Android tablet

Posted by

Running QGIS on an android phone worked just fine – but running QGIS on an Android tablet is going to be a lot more useful, with a larger screen. I summarised the steps of installing QGIS on a Galaxy Advance (I9070) in a previous post, under Gingerbread (2.3.6); since then, there have been some issues with QGIS breaking due to Ministro (supporting library) updates, so I went through the installation steps again, this time on an Asus TF300T running Jelly Bean 4.1.1.

Installation

  1. Enable “Allow installation of apps from unknown sources” – Settings|Security.
  2. Download the installer .apk (this download should  be fixed and up-to-date) and open it (on most Androids it will download to a “Downloads” directory, but may be elsewhere; you can just click to open the file from your file browser and it should start the installer).
  3. Run the installer and select ‘download and install’. 83 MB of data will get downloaded. Luckily, it is a resumable download – if it fails, you can start the process off again where you left off.
  4. Confirm that you want to install (standard Android dialogue) – takes about 2 minutes to finish this stage.
  5. Run QGIS: “Unpacking post-install data” … 10 seconds.
  6. QGIS requires a supporting service, available from the Play store “This application requires Ministro service. Would you like to install it?” Confirm and install Ministro II, only 523KB.
  7. Looks like it needs MORE libraries to run! “Qgis needs extra libraries to run. Do you want to download them now?” Confirm and install the QtCore libraries – a lot bigger at 31MB.

Does it work? Sort of …

Sure does … except for the Python plugins, which are not working, it does run and I’ve used it in the field a few times now. The version I’ve just installed, however, won’t let me select the panels to view, which means I can’t access the GPS connection panel to do live tracking. This really limits the utility, but one can pan, zoom and view maps.

Screenshot_2013-03-27-13-33-55Screenshot_2012-08-23-15-13-09 Screenshot_2012-08-23-15-22-14 Screenshot_2013-03-27-13-33-18

Tips and Tricks

  1. Export a QGIS project from your desktop into a single folder + .qgs file using the QConsolidate plugin, then transfer that consolidated folder + project file to the MicroSD card. This will gather all the rasters and vector files into one place for easy transport.
  2. Don’t work with high-resolution rasters unless you’re very, very patient; turn these off or remove them altogether. In the example above, the red ‘slope’ layer was a 25%-scale version of my original (which we just had to show us places we couldn’t fly low-level safely in the Cessna).

GIS Tips – Think about how you are going to present it

A map’s a map right? You get the data and you make everyone happy with a nice map….not always.

Chatting the other day to a colleague and they don’t like using 3D, the justification was that there isn’t any reason to, which (as you’ve already guessed) got me a little wound up. So, I thought I’d have a look around on the web and had a real moment of “why did they do it that way?”

At this point, I wish to point out that my way isn’t (always) the right way, there are many considerations to be made when finding the right media for your geospatial representation, mostly it comes down to what the client/customer is paying for but as GIS professionals we should be steering the end user from some of the monstrosities around and making the results easier to understand

Don’t just colour in the pictures (all the time)

I like to think that rather than a scientist, I am an artist. One day I hope to see my work on someone’s wall or for someone to ask me to create them a bespoke piece like one of my previous works –  and to a small extent we Geomagicians ARE artists, we may not always produce works of art but we have choice over our presentation medium and we can pick the right colour crayons.

Much like the sculptor, the oil painter or modern artist, we can choose, over we use 2D paper maps, 3D videos, web maps or something a little more interesting (3D printing anyone?!) and our choice of medium should be driven by providing the end user with the best way of understanding the mass of information collected.

I’m sure that I’m more Escher than Dali

Let’s run through a few ideas –

You’re asked to produce a map of a site layout for a new building. It’s for a presentation which will be seen by a variety of people across a project…..

Yes, a topographic map would look great. Assuming that everyone is map savvy, they can all see where everything is going…..BUT….what if you produced the layout as a little 3D video clip, where it rotated around the building being put in? Even if there isn’t much detail in it, it would provide EVERYONE, even the non map savvy to understand the context and relationship the features will have on the ground.

You’re asked to create an environmental map to show the issues on a particular site, what you didn’t realise was that there were 15 overlapping layers. This is going in a technical report and only has room for 1 page…..

How much hatching is there in the world? You could again, provide an intricate layered topographic map OR use an interactive PDF – why not use the “geoPDF” format and allow the end user to switch their layers on and off? Although cartographically you would still need to contemplate how the layers will display, you are not restricted to ensuring they are all visible on print.

Your boss wants to show on a presentation, the urban growth for a small village over the last 10 years…..

If you’re thinking that a nice Ordnance Survey based map with the buildings for each year in different colours, you might not get that promotion – why not either a) create a time slider map (available in QGIS & ArcGIS) or b) create a little video showing the change over time??

This is what I created when I was asked to show the UK wind turbine rounds (turn your speakers up):

You are asked to show how a shipwreck lies on the seabed as the bathymetry doesn’t match the photo in the book.

Why not extract the image from the book and georeference it over the top of the bathymetry? Then animate it as a 3D video so that the user(s) can see it from all angles? That’s what I did….this is one of the early images, the detailed images are licensed.

So why bother?

I’m not saying that you should give up paper maps and PDFs, what I’m questioning is our laziness as data visualisers. Sometimes we take the easy option rather than remember that we are the story makers or the answer providers….and what Jack Dangermond is missing is that we are responsible for TRANSLATING the story and information too.

Next time you get a request for a map through, instead of saying “yes” and churning out another identical map to the other 30,00+ you already have, take a minute to think about whether the end user is going to understand the results. Don’t assume that you know the end user, instead show the receptionist the map and ask if she/he can explain it or maybe show your partner….don’t assume that you can think like a non map savvy user!

For every unhappy or confused client, you will lose 10 more through word of mouth and bad press……for every happy client you will not only gain a great sense of happiness, but potentially another 3 clients.

Nick D

Friday Fun…where do my followers come from?

So that I could be a little more relevant, I thought I’d have a look at where my followers come from….should I be surprised that I have 1.4% following from Kiribati? Or that I have more followers from the USA than my home country? I am sure there is a map that should be made here somewhere……

0%    Afghanistan
0%    Albania
0%    Algeria
0%    American Samoa
0%    Andorra
0%    Angola
0%    Anguilla
0%    Antarctica
0%    Antigua and Barbuda
0%    Argentina
0.4%    Armenia
0%    Aruba
2.9%    Australia
0%    Austria
0%    Azerbaijan
0.4%    Bahamas
0%    Bahrain
0%    Bangladesh
0%    Barbados
0%    Belarus
0.4%    Belgium
0%    Belize
0%    Benin
0%    Bermuda
0%    Bhutan
0%    Bolivia, Plurinational State of
0%    Bonaire, Sint Eustatius and Saba
0%    Bosnia and Herzegovina
0%    Botswana
0%    Bouvet Island
2.2%    Brazil
0%    British Indian Ocean Territory
0%    Brunei Darussalam
0%    Bulgaria
0%    Burkina Faso
0%    Burundi
0%    Cambodia
0%    Cameroon
6.9%    Canada
0%    Cape Verde
0%    Cayman Islands
0%    Central African Republic
0%    Chad
0%    Chile
0.4%    China
0%    Christmas Island
0%    Cocos (Keeling) Islands
0.4%    Colombia
0%    Comoros
0%    Congo
0%    Congo, the Democratic Republic of the
0%    Cook Islands
0%    Costa Rica
0%    Croatia
0%    Cuba
0%    Curaçao
0%    Cyprus
0%    Czech Republic
0%    Côte d’Ivoire
0%    Denmark
0%    Djibouti
0%    Dominica
0%    Dominican Republic
0.4%    Ecuador
0.4%    Egypt
0%    El Salvador
0%    Equatorial Guinea
0%    Eritrea
0%    Estonia
0%    Ethiopia
0%    Falkland Islands (Malvinas)
0%    Faroe Islands
0%    Fiji
0%    Finland
0.4%    France
0%    French Guiana
0%    French Polynesia
0%    French Southern Territories
0.4%    Gabon
0%    Gambia
0.4%    Georgia
1.8%    Germany
0%    Ghana
0%    Gibraltar
0%    Greece
0%    Greenland
0%    Grenada
0%    Guadeloupe
0%    Guam
0%    Guatemala
0%    Guernsey
0%    Guinea
0%    Guinea-Bissau
0%    Guyana
0%    Haiti
0%    Heard Island and McDonald Islands
0%    Holy See (Vatican City State)
0%    Honduras
0.4%    Hong Kong
0%    Hungary
0%    Iceland
1.4%    India
0.7%    Indonesia
0%    Iran, Islamic Republic of
0%    Iraq
0.4%    Ireland
0%    Isle of Man
0%    Israel
0%    Italy
0%    Jamaica
0%    Japan
0.7%    Jersey
0%    Jordan
0%    Kazakhstan
2.2%    Kenya
0%    Kiribati
0%    Korea, Democratic People’s Republic of
0%    Korea, Republic of
0%    Kuwait
0%    Kyrgyzstan
0%    Lao People’s Democratic Republic
0%    Latvia
0%    Lebanon
0%    Lesotho
0%    Liberia
0%    Libyan Arab Jamahiriya
0%    Liechtenstein
0%    Lithuania
0%    Luxembourg
0.4%    Macao
0%    Macedonia, the former Yugoslav Republic of
0%    Madagascar
0%    Malawi
0.4%    Malaysia
0%    Maldives
0%    Mali
0%    Malta
0%    Marshall Islands
0%    Martinique
0%    Mauritania
0%    Mauritius
0%    Mayotte
1.4%    Mexico
0%    Micronesia, Federated States of
0%    Moldova, Republic of
0%    Monaco
0%    Mongolia
0%    Montenegro
0%    Montserrat
0%    Morocco
0%    Mozambique
0%    Myanmar
0%    Namibia
0%    Nauru
0%    Nepal
0.7%    Netherlands
0%    New Caledonia
0%    New Zealand
0%    Nicaragua
0%    Niger
0.4%    Nigeria
0%    Niue
0%    Norfolk Island
0%    Northern Mariana Islands
0%    Norway
0%    Oman
0.4%    Pakistan
0%    Palau
0%    Palestinian Territory, Occupied
0%    Panama
0%    Papua New Guinea
0%    Paraguay
0%    Peru
1.1%    Philippines
0%    Pitcairn
0%    Poland
0.4%    Portugal
0%    Puerto Rico
0%    Qatar
0%    Romania
0%    Russian Federation
0%    Rwanda
0%    Réunion
0%    Saint Barthélemy
0%    Saint Helena, Ascension and Tristan da Cunha
0%    Saint Kitts and Nevis
0%    Saint Lucia
0%    Saint Martin (French part)
0%    Saint Pierre and Miquelon
0%    Saint Vincent and the Grenadines
0%    Samoa
0%    San Marino
0%    Sao Tome and Principe
0%    Saudi Arabia
0%    Senegal
0%    Serbia
0%    Seychelles
0%    Sierra Leone
0.4%    Singapore
0%    Sint Maarten (Dutch part)
0%    Slovakia
0%    Slovenia
0%    Solomon Islands
0%    Somalia
1.4%    South Africa
0%    South Georgia and the South Sandwich Islands
0%    South Sudan
2.2%    Spain
0%    Sri Lanka
0%    Sudan
0%    Suriname
0%    Svalbard and Jan Mayen
0%    Swaziland
0.7%    Sweden
0.4%    Switzerland
0%    Syrian Arab Republic
0.7%    Taiwan, Province of China
0%    Tajikistan
0%    Tanzania, United Republic of
0%    Thailand
0%    Timor-Leste
0%    Togo
0%    Tokelau
0%    Tonga
0%    Trinidad and Tobago
0%    Tunisia
0.4%    Turkey
0%    Turkmenistan
0%    Turks and Caicos Islands
0%    Tuvalu
0%    Uganda
0.4%    Ukraine
0.4%    United Arab Emirates
15.2%    United Kingdom
48.2%    United States
0%    United States Minor Outlying Islands
0%    Uruguay
0%    Uzbekistan
0%    Vanuatu
1.4%    Venezuela, Bolivarian Republic of
0%    Viet Nam
0%    Virgin Islands, British
0%    Virgin Islands, U.S.
0%    Wallis and Futuna
0%    Western Sahara
0%    Yemen
0%    Zambia
0%    Zimbabwe

 

Oh…here it is!

Percent Folowers By Country

But why stop there? What I’d love to know is what the percentage of followers by Square Kilometres (kilometers if you are American) is…..

Folowers By Countrypersqkm

Nick D

 

Remembering the Father of GIS

This was posted on the ESRI Canada blog yesterday, 12th February 2014 and I can’t think of anything more suitable for such a great person;

Remembering the Father of GIS

Feb 12, 2014 / Posted by Joy Chan

Today, we honour the life of a great visionary. Dr. Roger Tomlinson, whom we fondly knew as the “father of GIS”, passed away on Sunday, February 9, 2014 at the age of 80.

Dr. Tomlinson invented the first computerized GIS back in the ‘60s, when he developed the Canada Geographic Information System for use by the Canada Land Inventory.

Thanks to his innovation, we can now easily overlay unlimited amounts of data on dynamic, digital maps and analyze information in numerous ways previously not possible. From climate change, overpopulation, poverty, disease outbreaks and flooding, to managing power outages, emergencies and optimizing site selection, GIS is being used today in various industries to help solve virtually any location-based problem.

Dr. Tomlinson’s invention of GIS led to the development of today’s computerized mapping technology, digitizing tables and global positioning systems. As well, his work advanced mapping as a profession and established a thriving industry that employs thousands worldwide. Research firm Global Industry Analysts estimates the global GIS industry is expected to grow to $10.6 billion by 2015.

He was a great friend to Esri Canada and an inspiration to many geographers. His work spanned over five decades and has helped organizations worldwide apply GIS to increase efficiency in map production, provide fast and easy access to digital data, and improve decision-making through visual spatial analysis.

In 1963, Dr. Tomlinson established Tomlinson Associates, a geographic consulting firm that served international clients including the World Bank, and the US and Canadian Forest Services. He was a sought-after speaker at GIS events worldwide and presented at several Esri conferences.

For his outstanding work, he received the Esri Lifetime Achievement Award (1997), the Royal Canadian Geographical Society’s Gold Medal (2003) and the National Geographic Society’s Alexander Graham Bell Medal (2010).  In 2005, he became the first recipient of  the Association of American Geographers’ Robert T. Aangeenbrug Distinguished Career Award.  He was made an Honorary Member of the Association of Ontario Land Surveyors in 2011 for his leadership in the area of GIS.  In 2001, he was named a member of the Order of Canada, the country’s highest civilian honour, and was promoted in 2013 to an Officer of the Order of Canada for transforming the field of geography with his invention of GIS.

Dr. Roger Tomlinson was named an honorary member of the Association of Ontario Land Surveyors at the 2011 Esri Canada User Conference in Ottawa.

Read more about Dr. Tomlinson and his work in this article: The 50th Anniversary of GIS.

A couple of years ago, I was fortunate to meet him in person at an employee event here at Esri Canada. Thinking of all that he has done for the GIS industry and the impact of his achievements on the world, I couldn’t help but be in awe of his brilliant mind and presence. Thank you, Dr. Tomlinson, for your remarkable legacy. You will be missed.

Since we heard of his passing, many colleagues have shared their own memories of meeting this industry giant and I wasn’t surprised that the single most common thing people share about their thoughts of him was his passion for GIS.

How did Dr. Tomlinson and his work inspire you? Share your thoughts and memories of him by leaving a comment below.

– See more at: http://www.esri.ca/en/blog/remembering-father-gis#sthash.YYGQWFJg.dpuf

GIS Tips – ArcGIS….I lost my Toolbar (and other toolbar problems)

It’s been a while since I did any tips and was asked by a colleague for some help yesterday which spurned me into putting this onto paper (or digital/matrix/whatever).

Have you ever been working away in ArcMap when all of a sudden, in your whirl of mousing and clicking, you accidentaly move or lose one of your toolbars? Yeah, me too. It happens to the best of us. So here I’ve a couple of ways to retrieve the toolbars easily.

1: The toolbar is just plain gone

If the toolbar has just completely disappeared, chances are that the little X in the top right corner has gotten clicked, therefore closing it. Right click anywhere on a toolbar and choose the toolbar you want from the list Go to View Toolbars and choose the toolbar you want from the list

 2: The toolbar has been moved partly off screen and you can’t reach the handle to move it back

You may think all is lost when your toolbar is just barely out of reach. Double click on an empty space on the toolbar. This will dock the toolbar, thus bringing it into view for you to do with as you please.

You might have to resize the toolbar in order to get an empty space to double click on, but it can be done!

3: You can’t reach/see all of the buttons on your docked toolbar

If some of the buttons on your toolbar are unreachable, it is likely because it has been docked on a spot that doesn’t have enough room to accommodate all of the buttons.

Move the toolbar to another docked spot with more room & undock the toolbar

4: Toolbars won’t stay where they are

This problem seems most common in ArcGIS 10, the solution I have found is this;

1. Start Menu –> Run –> type “regedit” (without quotation marks).
2. Expand HKEY_CURRENT_USER folder.
3. Expand Software folder within HKEY_CURRENT_USER.
4. Rename the ESRI folder to ESRI_OLD.
5. File > Exit.

The toolbars may not be docked the first time but once in place they will be docked on subsequent refreshes.

 

Nick D

Why transforming data in the UK is so important for #GIS

Previously I wrote about the need to use the OSTN02 transformation when working between British National Grid (OSGB36 – EPSG27700) and WGS84 (EPSG4326) but I am astounded at the number of questions over why & how. It should never be the case that a developer contacts you to ask why a transformation is needed, let alone require explicit examples. Here we are though, I have spent the last 4 months fighting for the OSTN02 (or even the Helmert) transformation to be available in 3 GIS software to no avail, only questions…..so this post is to remind everyone that GIS isn’t about pressing buttons and hoping the code works but understanding HOW the core geodetics work so that the GIS output is precise.

UK has more than 2 balls

Although to many of you, the British National Grid is the primary coordinate system, there is actually 3 national coordinate systems. One system is American but used in almost everything we do, the second is the European standard and the third is the British standard, why? Here I am going to try and explain simply enough for those who didn’t study at school…….

When I was a child I though the world was spherical, well, it’s not,  in reality it is a biaxial ellipsoid similar to the one below.

Biaxial Ellipsoid

Our earth rotates around the shorter axis (90° to the image above) and is not as perfect as the image above would suggest. If the ellipsoid was entirely perfect, creating a coordinate system would be an easy task, there would probably be 5 or 6 different systems for the whole world and they would be infinitely accurate.

Unfortunately this is not the case, there are bulges and gravitational fluxes which mean that there cannot be an accurate projected coordinate system for the world. We can have a fairly accurate global MODEL (datum), WGS84 is accurate to about 4cm anywhere in the world but this is only if you know how long a measure of Latitude/Longitude is where you are, this can be a bit of a pain if you have a complex polygon or are trying to plot the coast of a country……

So, the way we deal with this is to create “Local Ellpsoids” which provide a best fit for the areas. An example of this is the Airy ellipsoid (the ellipsoid upon which the OSGB36 datum is based) which is specific to the UK. The image below from Ordnance Survey shows how the ellipsoids can be used:

Best Fit ellipsoidYou can see how, on the left of the image above, the regionlly best fitting ellipsoid doesn’t perfectly match the earth but it creates a large region of best fit which allows for precise calculation.

Balls to Ellipsoids

Ellipsoids are smooth, and the earth, as we discussed is far from smooth. Therefore I am going to bring in the concept of height…..afterall, if we are going to talk about ellipsoids and bulges then we also need to understand how the ellipsoids are modelled to be as precise as they are, but how high is high? Have you ever wondered why your GPS is wrong? Where does it measure the height from? Actually how high is high?…..

If we want to measure height we need a zero level, a point we can all ground, this is a model known as a Datum. At this point I could start getting technical and discuss how gravity affects height and ramble on for another 3 paragraphs or so but that would probably confuse most of the target audience here, lets just reference that I told you so that I can’t be blamed for not mentioning it 😛

Ellipsoids? Datum? It’s ALL Balls!

Okay, there we have it, the quickest and most basic lesson in geoids ever….if this is going to be used for anything more than reference, I strongly recommend you read more on the subject,Ordnance Survey do a great guide which makes it easy for the complete noob to read & IT’S FREE: Ordnance Survey guide to coordinate systems

Before I go on to answer the question posed in this blog, I am quickly going to clarify the coordinate systems discussed so far:

World Coordinate System: WGS84 (EPSG4326)

GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
    PRIMEM["Greenwich",0,
    UNIT["degree",0.01745329251994328,

European Coordinate System: ETRS89 (EPSG 4258)

GEOGCS["ETRS89",
    DATUM["European_Terrestrial_Reference_System_1989",
        SPHEROID["GRS 1980",6378137,298.257222101,
        AUTHORITY["EPSG","6258"]],
    PRIMEM["Greenwich",0,
    UNIT["degree",0.01745329251994328,

United Kingdom Coordinate System: OSGB36 (EPSG 27700)

PROJCS["OSGB 1936 / British National Grid",
    GEOGCS["OSGB 1936",
        DATUM["OSGB_1936",
            SPHEROID["Airy 1830",6377563.396,299.3249646,
        PRIMEM["Greenwich",0,
        UNIT["degree",0.01745329251994328,
    UNIT["metre",1,
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",49],
    PARAMETER["central_meridian",-2],
    PARAMETER["scale_factor",0.9996012717],
    PARAMETER["false_easting",400000],
    PARAMETER["false_northing",-100000],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH]]

All of a sudden some of that text makes a little sense, doesn’t it!!

Ok, I get the point, 3 coordinate systems….at some point I’ll need to make sure there isn’t any error

So, finally, back to the original question and I hope by now it has become fairly evident why we need transformations in our lives. Most onshore and offshore survey in the UK is done in ETRS89, this ensures the best level of accuracy (and because GPS systems are ETRS89/WGS84 based) when recording coordinates. Unfortunately this does not marry well with British National Grid because they are 2 different datum (it only took me 5 paragraphs to make that point) – to remove any potential distortion and minimise error, we require a transformation.

In the UK without a transformation we would get errors from 50m to 150m, with a simple 7 parameter transformation we can obtain accuracy to about 5-7m. The 7 parameter transformation is commonly known as a Helmert transformation and, as the name suggests, provides 7 shifts to the coordinates to create a “best fit”.

HelmertFor those interested, tx, ty & tz are translations along the x,y,z planes and rx,ry & rz are rotations. This snippet was provided by Ordnance Survey.

To ensure the level of accuracy required for building housing or mapping real world issues, a more accurate system is required. The transformation system for the UK which transforms ETRS89 to OSGB36 is called OSTN02 (Ordnance Survey Transformation Network 02) – this is a GRID of transformations based on measurements and corrections placed all over the UK, thus it erradicates almost all distortion, with a level of accuracy to about 2cm anywhere in the UK (within about 1km offshore).

Some people might try to confuse you with the term “NTV2” this refers to “National Transformation Version 2” which is more to do with the format of the file itself (.gsb). The transformation can be downloaded here: OSTN02

I’ve got it but what do I do with it?

And here lies the million pound question – the .gsb format was designed to make it easier for software to adopt the grid transformation but only a handful of software currently supports it.

Obviously, professional software like Fledermaus, AutoCAD, ArcGIS & MapInfo support the NTV2 format but some more popular software which you would expect to support it, don’t, these include, GRASS & QGIS 2 (1.8 & 1.9 d0 support NTV2).

Now that you know the above information, you’ve got to ask yourself, how can anyone work on UK GIS data without OSTN02?!

Please feel free to carry on the discussion with me on twitter I am @dragons8mycat

Nick D