Just where are the standards nowadays? (rant)

I am sure I am not the only one, you have the mother of all projects to get some spatial analysis done on by close of business and you go to merge the data for a spot of spatial join action and……there is no consistency, in fact every single data has a different way of structuring its attribute data.

Okay, so maybe you won’t be merging your data together for a spatial join, only a complete noob would be doing that (hello :P) but the point is, where are the standards? You buy survey data and most of its own data uses different attribute labelling formats, you download some Government Open Data and you spend 90% of your time trying to work out what the columns are about…if they are labelled at all!.

This is the modern age, we are looking at big data and cloud based systems yet we can’t even produce 2 data which talk to each other, it is well, embarrassing!! The ironic (or frightening) thing is that I have sat in a few meetings and discussions on Spatial Data Infrastructure (SDI) and each meeting has its own ideas about what needs to be addressed – The hydrographers want something specific to them and then the survey guys want something specific to them….and so on…

Well there are many SDIs about but here’s the problem….too many chiefs and not enough Indians (excuse the analogy), there are too many rules and regulations from too many sources for anyone to make sense of it & without any single specific governing body we end up with everyone having their own ideas over what they think should be right….only most of these guys are analysts, theorists and conspirators, not the people that work with the geospatial data on a day to day basis.

Just a quick scout on Wikipedia reveals just how many different standards are flying around at the moment:

My question to you all is why can’t the field with the object names in be called “Names” or the field with the WGS84 Latitude coordinates (in Decimal Degrees be called “Lat_DD”? Is this too complex?

I’m looking at my Twitter right now and I see 2500 of the best people to answer my question, if not, to spread the word and help to find the BEST answer to the question –

HOW DO WE GO ABOUT GETTING A DEFINITIVE SDI FOR FEATURE ATTRIBUTION?Please feel free to prod me and poke a stick at me if I am completely wrong and there is a standard which should be adhered to, in 10yrs of geospatial work I haven’t found it but I may have been looking in the wrong places. The closest I have got so far is the amazing work being done as part of the Inspire directive, though trying to comprehend it is a real pain and a few of the attribute names are not compliant with shapefile format eg “meanhigherhighwatersprings” & “Gradeseperatedcrossing” is never going to work!

All I want to do is to inter-relate my geospatial data and not spend hours having to rename and re-order the fields so that it I can join 2 tables together! It can’t be that hard….can it?

Nick D

4 thoughts on “Just where are the standards nowadays? (rant)

  1. Hi Nick,

    INSPIRE is doing its best for pan-European data transfer, but faces a challenge:

    i) Try to do incorporate everything and it becomes too complex to gain a critical mass of adopters (save for European Government departments mandated to comply) – it is also reliant on domain experts in each sector putting their case forward as the “primary” (therefore de-facto standard) use case in developing the standard(s).
    ii) Try to keep it simple to enable the greatest number of adopters, but have the standard useful to no-one as the data provided will have but the thinnest veneer of attribution.

    I hear your complaint on field sizes, but this is a software vendor issue. There are no such challenges when using a Cadcorp .bds file for example. Unfortunately, whilst representing the greatest flexibility of the three, this file format is not nearly as widely used as the .shp and .tab file types of ESRI and Pitney Bowes MapInfo.

    Back to the standards – the IHO (International Hydrographic Organisation) got it right with S-57 standard for encoding nautical vector information and the emerging S-100 standard improves upon this – http://www.iho.int/iho_pubs/IHO_Download.htm. Unfortunately, their usefulness is relatively limited to the SOLAS (Safety of Life at Sea) market – but attributes are consistent when you bring together UKHO and SHOM data across the channel.

    In my days at SeaZone Solutions (both under Admiralty Holdings Ltd and later HR Wallingford Ltd ownership) we recognised this problem and took steps to correct it through the development of HydroSpatial II. Like you we attempted to combine thematic data (e.g. seabed sediments) but found little consistency across offshore national boundaries in the respective data models.

    Put simply, if field definitions do not match (or are not so close as to be considered consistent) then they represent distinct fields in any attribute combobulation. The net result is a table (attribute array) which is both vast and littered with null values.

    The only practical solution is for data providers to provide Product Specifications comprising a human readable Feature Catalogue and an Application Schema to pre-load the user’s spatial database and for testing once data is loaded. In each of the Feature Catalogue and Application Schema it requires sound definitions of the feature types, attribute types and attribute values (the latter where there are enumerated or list field types).

    SeaZone supplies a Feature Catalogue as a free/instant download – http://www.seazone.com/HydroSpatialBase.php – and provides an Application Schema with licensed material.

    It doesn’t solve your problem entirely but does go a long way to understanding what you have and enabling the you – the qualified user – to create data mash-ups only when the generalisations in doing so, are acceptable to that user.

    Regards,

    Andrew Iwanoczko

    1. Thanks for your response Andrew, ironically I find the SeaZone standard of data extremely high (except for the metadata…though this is a small gripe). The data is structured following the Inspire standard which I believe was originally defined by Dr Osborne himself is it not?
      I come from an industry which works across many areas, construction, survey, hydrography and even oil & gas…each has its own method and its own principles, unfortunately they don’t all conform.
      As you suggest, I use the Inspire standard as this covers many of the other standards such as MEDIN, ISO and OGC but it would be nice it there was only one which everyone HAD to conform to for any data transfer.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s