Open Source and Proprietary #GIS software – the eternal debate

Recently I posted on Twitter that there should not be a divide between software types, you should use the best for the job…to my surprise this started a rather interesting debate on not only business models but also how people thought of open source. So, what I’ll try and do is to clarify to those who are addicted to proprietary software why this is true and also help those addicted to free open source solutions (FOSS) why it is hard to get out of the “ESRI” mindset.

For the FOSS addicts (understand the why)

When I started out in GIS I didn’t get an option over software, it was ESRI or nothing, QGIS was in its infancy and the only options were clunky systems like the early Tatuk or the early GRASS. Installation as an open source noob was nothing more than mind bending with many being Linux based – you have to keep in mind that most geospatial specialists spend time understanding 3D geometry, not writing code…therefore just the simple concept of “compiling you software” was impossible.

Times have changed but I still believe that there is a little bit of a gap there…You are a geodetic guru and you want to set up your own business….what software do you choose? The one where the manufacturers will come to your door and install the software for you and answer the phone to any questions 24/7 or an open source solution which is heavily supported by a community of GIS geniuses?

Unfortunately in this day and age, this is the decision us GIS Consultants and developers are still facing today, we can sell and push for open source but business is not always about saving money, it is often about reducing financial RISK.

For the proprietary addicts (understand the why)

Over the last few years my job role has changed, not because I have become any more of a genius (definitely not!), it is due to the market changing. It used to be the case that clients wanted some nice map which showed the issues with their project and this map could be emailed around, everyone was happy. Now clients want web maps and the ability to interact with the data, to make live changes so that whole teams of users can be aware of the ongoing project issues – If you are using proprietary software you are looking at this and thinking “That’s going to cost you”…..and this why I believe FOSS is starting to become more common place.

Over the last couple of years open source software has done a ninja attack, the quality of the FOSS GIS software now is nothing more that remarkable. There is very little that proprietary software can do that FOSS can’t, including server systems. Yes, I said server systems, there are free solutions like QGIS Server and OpenGeo which are just brilliant! Even for a Geomatics type like me, I can easily set up a server system and (dependent on buying hosting) be putting web-maps up all over the place in under an hour.#

In this day and age the argument has diminished. Before, if I went to the board of a company and said that I was going to build an open source web mapping system, I would have to convince them for several months that the risk was worth it – but faced with the cost of the other proprietary server systems out there being above ¬£75,000 most companies would agree that a demonstration of the open source would be worth a go……and the open source options are fantastic, once the demo is running, they are hooked!

FOSS or Proprietary?

Well here we are, the million dollar question and I am sure my answer will throw me out of many circles that I am currently involved in. My job here though is to be the voice of the USER, not a FOSS user nor a proprietary user, a geospatial analyst/consultant/science/nerd trying to get the end user the correct answer.

The answer? – BOTH

If I have an issue with the geoid I am working with and have a deadline which is getting closer, I need to know that I can phone someone and get the right answer there and then. I have had issues with datum transformations and trilateration and every time I have had an answer from the person on the other end of the phone within a few hours and the project has been saved.

In this instance proprietary software is the answer. Of course FOSS like QGIS has commercial support but having had first hand experience, I have never had the turn around which I have received with the ESRI guys.

For proprietary software, the issues lie with the GUI and cost…..these guys have been the only option for a long time and they haven’t really moved with the times, the software has become clunky and sluggish in comparison to the FOSS, not only this, how can they justify the thousands of ponds they charge for software which is that slow and clunky? Extra costs for add-ons which provide necessary functionality which is all free and provided in QGIS? FOSS has a great community too, if I find that I DO have an issue (with the base system, not the plug-ins), there is usually an answer within 48hrs and furthermore the software is flexible and open, you can customize it in a million different ways to cater it to different needs.

In this instance FOSS wins purely down to its flexibility, speed and potential for expanding.

My conclusion is that to provide the correct answers and represent the results well, you need BOTH software at this time, though I do have a message to both parties:

FOSS – Not everyone is coders, when we come to you with a problem, please understand that most of us are the mathematicians, scientists and geographers who design the theory and don’t have the time or capability to learn new languages.

Proprietary Software – Be more open and don’t be so greedy! ESRI, your finest moment was ArcGIS 3.x, everyone in the GIS world still talks about it, there are still MANY users on this platform, why? Because it was flexible allowed for customisation. The current software is turning the average consultant to open source because it is no longer financially viable, it is now better to take the risk than to invest in such an expensive system.

The over-arching message to the reader here is – USE THE BEST SOFTWARE FOR THE JOB, make notes as you work and over time you will see trends in what you do…see if you can improve those weaknesses by using another software or by chatting to the current software provider. If you are starting out in GIS completely cold and have time to invest, I recommend QGIS, if you need to be up and running quickly and have risk associated with your output, choose ESRI. Whatever you choose, choose wisely as it will reward you later.

Now that I have ostracized myself from GIS, I hope that it is a help to other out there who are starting out or feeling the same way ūüôā

Nick D

#GISChat transcript for 20th November 2013

A huge thank you to everyone who took part in the first #GISChat, it was great to see such a wealth of experience and views.

For some reason (As Dale Loberger pointed out) my tweets were not wholly appearing in the feed, therefore apologies for the missing text from myself, I hope to have this rectified for the next event on 18th December at 9pm.

The chat began at 9pm (GMT) and I welcomed everyone and asked everyone to introduce themselves:

  1. Hello¬†#GISChat, I’m GIS Data Coordinator at Angus Council in Scotland.
     
  2. Hello¬†#GISChat¬†i’m the GI Engineer at Angus Council
  3. Hello¬†#GISchat! I’m almost a 30 year GIS veteran working in the private sector that whole time. Now focusing on public safety apps of¬†#GIS.
     
  4. Some of you may have seen the #geocake at Angus Council today? I think we over-catered but #GISday was very useful indeed#GISChat
     
  5. Two very enthusiastic secondary school teachers who need to teach kids about GIS Рdef going to follow that up #GISChat
  6. #GISChat we had lots of visitors throughout the day from Social Work. Legal people, teachers
     
  7. Then conveyancing team needing linked title deeds with property and housing records and integrated workflows #GISChat
     
  8. people were all internal staff and a mix of power users, sometime users and “what’s GIS?”¬†#GISChat
  9. @Dragons8mycat this was my first #GISDay not spent at an event since they started in 1999. #GISchat
  10. #GISChat We also enticed none GI people with #GEOCake and then hit them with questions and showed them how they could use GIS too
     
  11. @Dragons8mycat funny, but your tweets are not showing in the official #GISchat filter
     
  12. @Dragons8mycat at least three options to get more integrated datasets and more people using a single source of the truth#GISChat
  13. @Dragons8mycat past #GISDay we had a good mix between pros and novices. The public is hard to entice without some exposure to#GIS #GISchat
  14. @DaleLoberger Thats a pretty good run of GIS days. After todays success we need to plan earlier for next year and get more peeps in#GISChat
  15. #GISChat¬†Belated happy GISDAY. We didnt do anything today for this gobal GIS event. We’ll probably do something in a few weeks.
     
  16. @COsmitty¬†@Dragons8mycat¬†haven’t changed much. We still discuss “What is¬†#GIS?” every¬†#GISDay¬†#GISchat
     
  17. We also ran an easy geoquiz with a mix of geography and GIS questions – some designed to make use of web GIS to get answers#GISChat
  18. @mixedbredie hated to break a perfect record, but like@Dragons8mycat I had to work. #GISChat
     
  19. @geosmiles were you out playing with QGIS on your tablet? you going to sell that to field workers? #GISChat
  20. @Dragons8mycat you know that is my favorite subject! #GISChat
  21. @DaleLoberger @Dragons8mycat Really? Rinse and repeat. Pretty maps & awesome analysis #gischat
  22. Q: How big is a Scottish rod? Quite surprised a few people new that straight out. #GISChat
  23. @mixedbredie #GISchat took the tablet up to the tree section and they loved it. Even the address management team wanted a bit of it!
  24. @mixedbredie It also embarrassed a few too! #GISChat
  25. @COsmitty @Dragons8mycat it is often the pretty maps that overshadow the great analysis. #GISchat
  26. @COsmitty @Dragons8mycat it can be a real challenge to get the effort behind getting the question/answer right #GISChat
  27. @geosmiles @mixedbredie #GISChat A mobile device would have been handy when I was Address Custodian at Dudley, had to use parchment & quill
  28. @Dragons8mycat¬†@COsmitty¬†…SHOULD enhance…¬†#GISChat
     
  29. @reagarbett True dat. The country with the highest minimum elevation caused much discussion too. #GISChat
  30. @mixedbredie¬†I sent my answer from the wrong account and forgot#GISChat¬†anyway – isn’t a rod 16.5 feet like in US?
     
  31. .@Dragons8mycat¬†sorry busy busy ūüôā will try & join in if I can#gischat¬†#gisday
  32. @reagarbett¬†@geosmiles¬†just recently we have had two requests for mobile GIS to capture street furniture – we need a tablet…#GISChat
  33. @DaleLoberger @Dragons8mycat Shiny object map and analysis hide ugly data. #gischat
  34. @Dragons8mycat #GISChat the right name for #GIS is a topic everywhere Рin fact #EMS is asking if that is the right name there too.
  35. @COsmitty @Dragons8mycat definitely. Maybe that is why I like to get better handle on the analysis than worry abt the map #GISChat
  36. @Dragons8mycat #GISchat it is a samsung 10.1 with wms to mastermap.
  37. @DaleLoberger @COsmitty @Dragons8mycat but the masses pay attention to pretty maps- lesson needs to be learnt #gischat
  38. @DaleLoberger @COsmitty @Dragons8mycat for sure once you know the data and understand the analysis the map prod becomes clearer #GISChat
      
  39. @mixedbredie #GISchat we contracted someone to do this for us about 18 months ago, produced some really good datasets.
  40. @mixedbredie @COsmitty @Dragons8mycat they certainly hide less that way #GISChat
  41. @Dragons8mycat @DaleLoberger no separation needed. I prefer to start as a geek and pick lens according to project or phase #GISchat
  42. @Dragons8mycat We had a small gathering here in Dublin, a few well known people and a couple of newbies. #GISChat
  43. @Dragons8mycat #GISchat peeps not really that interested in Mastermap attributes, its just something to navigate and plot by!
  44. @Dragons8mycat @COsmitty this is where I separate cartography from GIS (which I see as primarily the analysis) #GISChat
  45. @AngharadStone¬†bit more pressure on us then to get the map to give the right message. Lots of diff hats to wear…¬†#GISChat
  46. @AngharadStone¬†bit more pressure on us then to get the map to give the right message. Lots of diff hats to wear…¬†#GISChat
  47. @Dragons8mycat It was a bit last-minute, in fairness. #GISChat
  48. @Dragons8mycat They work with one of the regulars. Had some real out-of-the-blue types at another #GeoBeer last month tho.#GISChat
  49. @COsmitty¬†@Dragons8mycat¬†how many GIS people know “effective cartography”?¬†#GISchat
  50. @DaleLoberger¬†explain “effective cartography”? simplest way to convey message?¬†#GISChat
     
  51. Late to the¬†#gischat¬†– hello all, I’m in Euston after the¬†@foss4g¬†wrap up and catching up
     
  52. @mixedbredie¬†“clearest” way to convey the message without confusion¬†#GISchat
  53. @mixedbredie @DaleLoberger #GISChat would it be the best blend of background noise to produce the image?
  54. @mixedbredie @DaleLoberger Effective cartography- the ability to communicate a story, results of analysis, or focus on data #GISChat
     
  55. @COsmitty @daleloerger @reagarbett often the bit I find hardest and then I turn to map galleries for inspiration and fresh eyes#GISChat
  56. @Dragons8mycat The last official gathering of the @foss4g 2013 team, capturing lessons learned and tidying up a few things #gischat
  57. @DaleLoberger¬†@COsmitty¬†@Dragons8mycat¬†Worked alongside Graphic Designers in my early years, an eye-opener re: ‚Äėeffective‚Äô#GISChat
  58. With Foss4g in the UK this year there was a lot of open on the agenda, what are the topics for 2014? #gischat
     
  59. @COsmitty @DaleLoberger luckily I can just turn to @mixedbredieand say does this look right to you? #GISChat
     
  60. @DaleLoberger @COsmitty @Dragons8mycat Main learning: a good looking map can hide a multitude of sins #GISChat
     
  61. @abi_csgn heading north on the sleeper or the cattle trucks?#GISChat
     
  62. @reagarbett I miss that Рjust me covering GI & Data management#gischat
     
  63. @ManAboutCouch¬†@COsmitty¬†@Dragons8mycat¬†…or a multitude of “lies”¬†#MarkMonimer¬†should join¬†#GISchat
  64. @abi_csgn #GISChat late night travelling? were there any#GISDay celebrations @foss4g or GEOCakes?
  65. @abi_csgn @reagarbett ah yes from lone wolf to team of 7- different challenges. #gischat
     
  66. @Dragons8mycat @abi_csgn @foss4g #GISChat next year conference is in Portland.
  67. @abi_csgn¬†I’m sure if you send¬†@mixedbredie¬†a screen shot, he’ll provide the same service¬†#GISChat
  68. @mixedbredie¬†yup, sleeper for me, early start this morning will hopefully mean I sleep through the bumps…¬†#gischat
     
  69. @mixedbredie anyone archiving or storifying #gischat or is this less formal?
  70. @DaleLoberger @COsmitty @Dragons8mycat That book gets more relevant every year. Hard to believe it’s 22 years old! #GISChat
     
  71. @COsmitty¬†@abi_csgn¬†@reagarbett¬†I did solo GIS for 18 months before Roger joined – don’t know how I managed…¬†#GISChat
     
  72. @COsmitty yes, I am sure the other way is equally interesting#gischat @reagarbett
  73. @COsmitty¬†@abi_csgn¬†#GISChat¬†I can image the project meetings may last some time …
     
  74. @Dragons8mycat I probably need to find a more effective way to track the #GISchat
  75. @COsmitty¬†I am sure¬†@Dragons8mycat¬†has a handle on that – I’m finding it hard to keep track of the threads – needs colours#GISChat
  76. @abi_csgn @mixedbredie Hey at least that is better than the MegaBus I did 2 weeks ago 10 hours travelling #GISChat
  77. @COsmitty @mixedbredie @reagarbett building strong external networks is key, more so when you are on your own #gischat
     
  78. @mixedbredie @COsmitty @Dragons8mycat maybe we could track#GISchat on a map?
  79. There must be a way of linking all the replies into a timeline in this twubs #GISChat
  80. @hp_ems Yup Рcan do that with FME and a couple of transformers but it only works on located tweets #GISChat
  81. @Dragons8mycat @mixedbredie @COsmitty @DaleLobergerWoohoo Cricket starts soon #GISChat #Ashes sorry just went off piste!
  82. @mixedbredie¬†@hp_ems¬†shouldn’t every GIS’er opt to geo-tag their tweets?!?¬†#GISChat
  83. It is official- profile corrupted. ArcMap on strike. No GIS for me on#GISDay Poetic. #GISChat
  84. @abi_csgn Indeed Рglad I joined AGI and QGIS User Group and the Scotland crowd are good at getting together too #GISChat
  85. Twubs makes #GISChat much nicer.
  86. @DaleLoberger¬†Maybe. Some people do get concerned about locational privacy…¬†#GISChat
  87. @mixedbredie @abi_csgn The Scotland #OpenStreetMap user group look v. active, even had their own SotM. Lessons for us in Ireland #GISChat
  88. @Dragons8mycat¬†@abi_csgn¬†@COsmitty¬†@mixedbredie¬†I’ve been driven batty by bat files, but would not be without them now#GISChat
  89. @mixedbredie my suggestion was *somewhat* tongue in cheek, but I do share location #GISChat
  90. #GISChat calling it a night, my cat is wanting the laptop!!!! goodnight all
  91. are you listening @fozy81 @osmedin @chrisfl ? MT@ManAboutCouch: The Scotland #OpenStreetMap user group look v. active, #GISChat
     
  92. @Dragons8mycat Thanks for setting up this #GISChat. I look forward to more!
     
  93. @reagarbett¬†see you next time…¬†#gischat
  94. till the morro all Рthanks Nick! #GISChat
     
  95. @Dragons8mycat¬†#GISChat¬†Portland Oregon! Must stop this thing auto correcting to Poundland…
  96. @abi_csgn @fozy81 @osmedin @chrisfl missed that chat, but maybe next time. #GISChat
     
  97. @Dragons8mycat @mixedbredie @abi_csgn I compared OSM editor numbers Sco/Irl with @chrisfl at #SotM13: v similar.#GISChat
  98. @fozy81 @abi_csgn @osmedin @chrisfl next #GISchat is Dec 18
  99. @Dragons8mycat @mixedbredie @abi_csgn Our OpenData regime is different tho, positive noises but no action yet from@OrdnanceIreland #GISChat
     
  100. @Dragons8mycat¬†@mixedbredie¬†I’ll ask the SMEM folks which tools they are using for archiving chat¬†#GISChat¬†could be fun
  101. Thanks all involved, keeping me occupied while hanging around Рsee you next time #gischat
    As stated above, next chat is 18th December, do join in!!

GIS Day is here! Be part of the community (its free)

Wednesday 20th November 2013 – GIS Day is here! The internet is alight with people wishing each other a happy GIS Day, but what is GIS day?

word_cloud_gis_26_10

The first GIS day was held in 1999, predominantly promoted by ESRI (who are still strong backers) it was an opportunity to open the doors to the wider world, to let others know exactly what it is us geo types do….sure, my wife will tell you I make maps but that really isn’t what it’s about and that is what today is about for us professionals.

For us professionals, GIS day is a great way for us to see how we can apply solutions to real world problems and demonstrate just WHY our service is so important.

For the academic, today is an opportunity to understand how the pieces go together, they may have spent months studying about the geoid but how is that applied in the real world?

Don’t forget that there are also many managers and colleagues who hear you talk about this magic “GIS” but don’t have a clue what it is, today provides the perfect opportunity for you to demonstrate your superior intellect and show them how you can track them in realtime…okay, so that may not be such a good idea, today is a great chance to show them how to work with you to improve your service and how you can help them with their work more.

Census_Bureau_Geographers_Cartographers

Most of all, GIS Day is about community, it doesn’t matter what software you are running, whether ESRI or QGIS, Mapinfo or GRASS, CADcorp or Tatuk….today we are all working on solving geospatial problems on the same globe, no matter what the datum. Today is the time to say hello to those around you and ask what it is they do…..you may find the solution to that issue that has been bugging you for months is a single question away!

We are all getting together on Twitter at 9pm tonight using the hashtag #GISChat – Feel free to join us ::

Geohugs all around

Nick D

20th November 2013 it all changes….. #GISChat starts

Image

Us geospatial types love a geobeer or two….maybe four or five, okay, go on, seven or eight….but even more than a geobeer, we love to get together and discuss all things geospatial and share our knowledge with the outside world. This is why, every year in November, we celebrate all things geospatial by having GIS Day.

This year to bring the geocake to the party, I am starting, with the help of Angharad Stone, Alan McConchie, Ross McDonald & ESRI UK,  a monthly meet up for all GIS users. Whether you are a new user, or an old user, a user who has lost their way or someone who is interested in GIS, no matter what platform, I hope to make this a chance for us to all catch up and chat once a month in the same place.

Image

Each month we’ll try and pick a topic matter and keep this as a main thread, though please feel free to branch off with any conversations which may arise! Ideas which have been suggested so far have been:

“Where do you see GIS in 10yrs time?”

“What has been the best advice you have received?” / “What one thing would you teach a new user?”

“Is GIS the correct term for our profession? If not, what should it be?”

As the first #GISChat lands on a rather special and monumental day, GIS DAY 2013, the first topic will be:

“What have been you experiences of GIS Day?”

Over the coming days I will add another page to this blog so that we can keep some form of transcript & have a place to discuss topics and ideas. – UPDATE: It’s now here

To join the conversation, all you need is a Twitter account and search for #GISChat РI look forward to chatting with you on GIS Day!!

Special thanks go to Alan McConchie who runs the fortnightly #Geowebchat (next on 19th November)-  please join in and be part of the growing community.

Nick D

Why is NTV2 important or … why you need your balls checked

This post is dedicated to a dear colleague of mine, who I have only worked with for a year but has had a profound effect on me and the way I look at my work. His name is Stuart Leather and he’s off to work for another company but he has left a phrase in my head that will never leave:

“Always check you balls”

Wise words, I think as we have become more digital we have forgotten (or at least not double checking) our “balls” or as I prefer to call them, ellipsoids & spheroids.

This has really come to light for me over the last few months where I have been helping people with the new QGIS 2.0 and also working on some web map development. Those of you lucky enough to follow me on Twitter will have seen my constant tweets about NTV2 support for QGIS 2.0 and also questions over how this can be remedied…..what was more surprising to me was that I have over 2500 followers and only a couple people had a clue what I was talking about.

More worryingly, I recently posted a question about NTV2 & grid transformations on GIS StackExchange and although I got a couple of half sensible answers, I was shocked to read the suggestion that I didn’t need a grid transformation from WGS84 to NAD83 as they were the same spheroid……

So….by this I am guessing that the GIS world has gone digital and has forgotten that they have balls, let alone check them.

Okay, for the sake of Stu, I am going to try and explain in very simple terms why you need to go get your balls sorted.

When using a single geodetic system (ball) it is relatively easy to switch between zones or different transformations as it only requires calculation based on a constant, hence why changing between WGS84 & WGS84 UTM Zone 30N is simple.

So why is it so much harder between WGS84 and OSGB36 or WGS84 and NAD83? Yes, you got it! It’s all to do with your balls, WGS84, OSGB36 & NAD83 all use different spheroids and these are not not even similar to each other, I could write a nice chapter here on geodetics but I am hoping that you know the reasons and issues here….essentially to get the best calculation between the balls you need a GRID transformation, this will allow for the variances between the balls, providing, in most cases, accuracy between balls of about 2-4cm.

You don’t always need this level of accuracy, if you are working on a 1:50k map, you really wouldn’t need to worry that much, so in this instance you could get away with a much simpler Helmert transformation (also called a 7 parameter transformation). A Helmert transformation, you may have seen it in your ArcGIS under something like “WGS84_to_OSGB36_Petroleum” will normally give an accuracy of about 7m between data.

So, will the world end if I DON’T check my balls?….well it won’t end but your career might….you see, depending on where in the world you are working, you could see differences between 50m and above (200m in the further extents of the UK between WGS8 and OSG36), so designing that coffee shop to sit next to the local cake shop might all go dramatically wrong…..

What I would like to see is for GIS developers to see this and realise that there is not enough provision made for those of us who need to provide a level of accuracy in our work or at least work between land & sea in the same map. OpenGeo have made a great start and allow the use of NTV2 grids and NADCOM grids within their web system. I have struggled to get PROJ4 and GDAL to work correctly with grid or helmert transformations, this means that transformations need to be applied to data (usually using ESRI ArcGIS) BEFORE inclusion to data before inclusion to most web map systems.

Thanks Stu for all your help over the last year, making me love my balls again. Good luck helping others with their balls.

Nick D