Last week, April 4th – 6th, one of the largest gatherings for the offshore industry took place, Ocean Business, in Southampton, UK. The bi-annual event is a large deal for the maritime industry. When I say large, I actually mean huge; there are over 350 exhibitors from 26 countries and 166 training and demonstration workshops in just those three days!
What is amazing about Ocean Business is that the whole maritime industry is represented. Of course there are the surveyors, the geologists, and the equipment hire companies, but also there are representatives from the GIS, CAD and engineering sectors all sitting side by side.
At the last event talk was about the latest Reson multibeam device. Whereas before conversation would have been around capturing scour and wreck details, this year was talk of analysing pipelines for cracks or whether bolts were where they should be. It was hard at times to remember that I was at an offshore event as the technology has developed so quickly.
Of course, in two years, technology has shrunk. With the onshore industry, this is not overly exciting but offshore this is BIG news. Take, for example, the Planet Ocean micro AUV, developed with the National Oceanographic Centrre. This micro-UAV is only 50cm long and shaped like the traditional “torpedo”. Being fully autonomous allows it to capture data in ways never previously considered.
Big improvements to Doppler Velocity Log (DVL) technology was demonstrated by most of the equipment exhibitors like Sonardyne, Nortek and Kongsberg. If you don’t know what a DVL is, it how you get location when you are underwater and have no GPS/GNSS. The systems on show were capable of capturing the location at better than 0.01m at depths up to 6000m. Just think that over and relate that to your super cool iPhone!
The data providers and geospatial companies who were present were murmering about BIM and how it is going to start to become an issue. How does BIM integrate with the information captured and provided offshore? Can the maritime industry build a common spatial data infrastructure?
One company I met that were really making strides on this was OceanWise. Dr Mike Osborne, a respected hydrographer, has taken it upon himself to get the maritime industry thinking about these issues and working towards a solution. OceanWise provides nautical geospatial data but also trains the offshore and onshore industries in the use of GIS for nautical charting, including maritime standards. Mike has set up maritime spatial data infrastructure groups across the world with 33 countries already involved, something which is unheard of in the onshore industries. Mike has agreed to be interviewed by xyHt, so look out for that in a later issue!
My final comment is about how much Lego was on display. I mean, do these guys have nothing better to do with their time? As much as there was UAVs, transponders, ROVs and new vessels, there were Lego survey vessels and tech remodelled in Lego. Maybe it is because I wasn’t allowed to take one home that I am so curious about it.
It’s no big secret, anyone can make maps, ESRI even had some children showcasing their abilities at the user conference this year, So why employ a GIS specialist? Well, apart from getting the cartography right, the specialist can advise on the legalities of map & data production.
With data becoming ever more available and online maps being the norm, there is now more copyright infringement than ever before. In some workplaces it is common to see Google Earth imagery & Streetview imagery put into reports as screen grabbed images….you do know that this is a breach of use?
With everything being so accessible nowadays it is easy to see something relevant and screen grab it, then make it available to others, but have you ever stopped to think about whether you can?
It doesn’t seem like a large number but 13% of all Government Google Removal Requests have come from court orders for copyright offences (figures here) & there have been over 27,662,690 URL removal requests to Google due to copyright (figures here). Okay, so a lot of the URL removals are music related, music piracy is a huge problem. But this highlights the breadth and size of the issue at hand and the rate at which companies are being reported and fined.
Law is a complicated beast, so to help simplify things I put together a flowchart to help end users come to the right decision.
Flowchart of the questions you should ask before using data or maps
Although it does not cover the plethora of data or map licence types, this chart provides an easy reference as to whether you may or may not use the material you intend to use. Of course this may vary from country to country and on a case by case basis, also this does not serve as a legal document, legal advice should be obtained in case of dispute.
A while back (names will be changed to protect the not so innocent) I was at a conference and bumped into an ESRI employee & introduced myself as Dragons8mycat, they responded by saying:
“You’re that guy that hates ESRI aren’t you?”
This got my heckles up and is what is fundamentally wrong with the GIS industry at the moment, it’s like a war, you are either in one camp or another and this should NEVER be the case, to get good at something you don’t read just one source, you don’t stick with one TV channel do you?…..
No, to learn and perform the best we can in life, we use MANY different sources. So why am I ridiculed for not liking parts or the services of software? Let’s be hones here, every single GIS on the market has its issues; ArcGIS crashes & is very pricey, QGIS (although 150% better than it was 3yrs ago) has a complicated GUI and lacks some basic features, CADCorp lacks proper 3D, MapInfo has transformation issues…….let’s not get me started on Tatuk.
At the same time there is a lot to love about every single one of those software and to get the job done I would have every single one of them on my machine; ArcGIS can do ANY geoprocess I need to do, QGIS is fast, reliable and integrates well with web development, CADCorp is fast, easy to use & has amazing support, MapInfo….is MapInfo…..but you can see where I am coming from.
Instead of fighting over which is best, segregating those who dare complain & creating systems with amazing features that turn back time or generate maps by retina recognition, the creators should look at what people are saying and fix what people are complaining about.
Excuse me for saying this but it is an industry in-joke that ArcGIS crashes constantly, since ArcMap 8.x there have been calls for an autosave feature, so why, at the pinnacle of GIS technology is there still online talk of constant crashes?……
Before you ask, I am impartial, I love all the GIS out there at present, each one of them has something great to shout about, if someone asked me what they should have in their office, at present I would be hard pushed to choose one, it would be a business case decision…..
This is me begging those developers and managers of the major GIS companies out there to stop bickering and take a look at yourselves, take the knocks on the chin and listen to what the public are saying (and not the bias devs around you).
I love you guys, all of you, but you’re giving me a headache…if you carry on like this someone is going to build a system that ticks all the boxes and I need you in my life too much to let that happen.
Little baby “Kit”…that’s his name and not a DIY project
I’m back! Yes, I know, I said that a month ago but baby number 4 arrived as well as building a new intranet map system for the company I work for AND writing articles for a magazine set to take over the world – xyHt.
Building your own web mapping system has become a right of passage in the GIS world, it’s easy to buy something off the shelf like ESRI’s ArcServer but how does it really work and there is still a lot of learning to be done with cost being related to “number of cores”, “spatial database format” & my personal favourite, “server OS” (I mean, a server is a server right?!).
I went down the ESRI route and it is a phenomenal system, when it works, it is unstoppable and you have full support to answer all those crazy questions but at a huge cost, when I moved jobs to a smaller company and they wanted an internal mapping system & told them about ArcServer they laughed……in hindsight I can see why now, £100,000 for a web map system for 40 employees is a bit OTT.
As a sort of cleansing of the soul, I thought it might be useful for those of you entering into the world of building web maps to share my love/hate of web maps so far……
Mapserver or Geoserver….that is the question
Okay, lets clear this up right from the start, if you are building a web map system, you are going to be looking at open source. Sure, there are other methods like Mapbox, CartoDB, QGIS Cloud, LizMaps and others but they mostly promise a free system for web mapping but then give you (a rather meager) 50mb of space or don’t include the tools you need, this is better for the smaller business but at the end of the day you will feel cheated….for full control you really need to build it yourself from scratch.
Your first map will most probably look like this…the infamous “pink tiles”
When I first started out with this web mapping 6yrs ago, there were 2 options if you wanted to create a web map, Geoserver or Mapserver. Both work with Windows & Linux, both are well developed and both are open source, completely free and extremely develop-able with some great tools included out of the box, the problem I had was choosing one….
On paper, Geoserver and Mapserver are pretty identical, both publish geospatial data from almost any format and have huge communities who can provide help and advice. If you get really stuck, there is also commercial support for both systems. The primary difference is the choice of programming language, although both systems use a combination of languages for certain tools, Geoserver is Java based whereas Mapserver is C at its core with a lot of PHP.
Okay, so let’s roll back, already we are talking programming languages! To get on the web you need a way of getting the data into a format that can be read by the internet, to do this you need a server, for maps you need a geo/map server, something that takes the shapefiles or dxf files and gets it onto a html page which can be put on a webpage somewhere.
A simple explanation for the map server
As you can see from the above diagram, the web server is the “converter” to get the data onto the internet, the data is “served” from the computer/server onto the internet. The format the data is most commonly used is Web Map Service (WMS) format.
Out of the box with both Mapserver and Geoserver you will get a server (Jetty/Tomcat/other), tools for conversion of data built in (GDAL, fTools), a LOT of projection systems (Proj4), OpenLayers (a web interface tool) and a demo map. Both systems initially install to your “LocalHost” which is a local network for your computer (it can’t be seen by anyone but you, though you will have a http address like http://localhost), to get the web server to be seen by the outside world you will need to adjust the settings in the systems server config to read the same as your outward going port connection (This may be worth checking out)
Mapserver or Geoserver….that is STILL the question….
So, going back to the Geoserver v Mapserver question, my first choice as a complete noob was Mapserver. Why? Because it sold me on its integration with QGIS, see my previous posts on how awesome QGIS is, essentially you can build your map in QGIS and then hit the “export to Mapserver” button and you have a webmap….if only….. Don’t get me wrong it is 100% better than building the map yourself but to a noob I assumed I could just use shapefiles and rasters and shunt it all to the internet with a shiny big button.
THERE IS NO ONE CLICK BUTTON
Once the Mapserver is installed and QGIS is up and running I stupidly expected to have a fully web mapping experience which would take over the world……not quite, QGIS provides you with a .map file which you then tweak and then you need to build a html page and the map tools (openlayers) around it. I may have spent my youth programming my Sinclair ZX80 & ZX81 but this was a huge step to start understanding how CSS & HTML worked and the OpenLayers (I’ll refer to it as OL from here) worked.
Tip 1: Firebug is your friend – you’ve probably seen a little picture of a bug in the top right of your screen when using Mozilla Firefox, this is a great tool for exploring the code which makes up webpages. Turn it on and waft your mouse around the page & you can see how bits of scripts work (or don’t work).
I am not afraid to say that my first map resembled something worse than a bad ZX Spectrum game, the tools didn’t work properly due to bad links, some of the layers didn’t draw up right and worse of all I felt completely out of my depth….I would post questions on the forums and it would appear that the questions I had were SO BASIC that they were mostly ignored…..
After 1 month of developing what I thought would be a new Google Earth, I put it on hold to think things over.
Don’t give up, it’s not as bad as you think
After a couple of months of chewing it over and feeling like a failure, I got back on the horse & read a couple of tutorials on the QGIS website written by the rather amazing Anita Graser. I’m not ashamed to say that I have a bit of a fan-crush on Anita, I contacted her on Twitter a few times when trying this system and she was so patient and really helped to make it easy to get going. Needless to say, after another couple of weeks of trial and error, I had a web map up and running.
Tip 2: Don’t be afraid to ask! Sites like GIS StackExchange are supported by some of the top people in the industry with some great advice. Even the dumbest questions get answered, we’ve all been there!!
Tip 3: Download and install PostGres & PostGIS – PostGIS is a spatial database system, much like the ESRI geodatabase, which is SQL based & is a more efficient and flexible way of storing spatial data. It is the most common way of storing data for web mapping & you will find it easier to work with this format in most systems.
A change is as good as a rest
Although I was having success with Mapserver, you always get that niggling thought that the grass is always greener on the other side, so decided to try Geoserver.
Much like Mapserver, Geoserver works straight out of the box on the localhost and has a nice interface for importing and managing the data.
Geoserver interface
To be honest, although the Mapserver had an easy system for making a map in QGIS, this interface made it easy to understand what was going on. I could load up layers, apply styles (in SLD format) and then use the layer preview to see what they would look like. The only downside? There is no map system integrated into it, so it can server WMS, WFS & others but you need to have a little HTML know-how to build it all into a map.
Tip 4: In Geoserver, layer groups are your friend, with these you can either create a basemap by grouping several layers together OR you can server a simple static map by using the wms direct in your browser
Tip 5: Static and Slippy…..There are 2 different types of web map, a static map which is nothing more than a picture or an interactive map which is known in the industry as a “slippy” map.
The great thing with Geoserver is that if you DON’T want to build a web map and just want to supply layers via wms to people, it is near perfect! The wms feed can be used within most GIS systems like QGIS, ArcGIS, uDIG, CadCorp and GRASS, to name a few. Again, the time and effort with this system is mostly spent designing the web map interface, calculating where tools need to sit, getting the legend to expand properly or getting the frame size just right with the company logo in the right place.
Lessons learned
The one thing I have learned from all this? Don’t be afraid to dive in, everything I learned from playing and trying to build in Mapserver, although in a different language, was similar in structure within Geoserver. The components and way the core works is essentially the same, it’s just deciding what you prefer in a system.
Once you have started to build one of these systems you begin to respect and appreciate the complexities of software like ArcGIS & QGIS but at the same time you start to understand how it all fits together. Although a stressful and humbling experience, it is one I recommend to any GIS noob.
Neo: “I know Kung-Fu” Morpheus: “SHOW ME”
Why go through all the pain?
Okay, so here is where I unveil the ninja-tricks, why go through all the pain of the above? What if there was a system which had the awesome interface of Geoserver but also integrated with QGIS, automatically installed PostGIS AND provided a map interface with a plethora of useful tools….all open source with the safety net of support (at a fantastically cheap cost) if you need it?
It’s true, I kicked myself 2yrs ago when I found OPENGEO SUITE by Boundless, although it has its quirks, it takes all the pain out of everything. Sure, there is some configuration to do, some of the tools like the print and CSS styling don’t work out the box but that is what people like me are here for (contact me for installs of OpenGeo or help adding tools).
First off, this isn’t a sales pitch, although I install OpenGeo Suite systems, I do it because since I found it, I want the world to use it….you could spend hundreds of thousands of pounds on a web mapping system which has a GIS front end and a web map output. OpenGeo Suite really is as simple as creating your map in QGIS (or ArcGIS then using GeoCat bridge) and then exporting it to Geoserver, you then use their GeoExplorer to show the layers.
Standard GeoExplorer interface
The point is?
Building your own web map system as a noob is scary and complex, lots of new terminology & methods. The developer & programmer types will bombard you with questions about your tile caching or getfeatureinfo whereas the average user will be asking whether they can upload their new NetCDF file.
Your experience will probably not be as bad as mine, things have developed FAST! Mapserver is now part of QGIS and there is a “QGIS Server” which is Mapserver based & Geoserver is now in version 2.5 – There are now so many users on Twitter & StackExchange that getting help is very easy.
My advice? Build a solution from scratch as a hobby, spend a little time every evening having a read and a play with either Geoserver or Mapserver….try and put your own map on a localhost system. When you can do that, install OpenGeo Suite and dazzle your employer!
