Tag: 3D

The Problem with 3D GIS

dragons8mycatLeave a comment

3D GIS is game changing, it can change the way you view your analysis, it can provide insights which you may have overlooked….but there is a little problem which I may had not shared in my deluge of blogs about how great 3D GIS is…..

…Sometimes it can be hard work. What I mean by this, is that it can sometimes need a lot more consideration than standard methods. Let’s look at some of the major culprits when using Esri ArcGIS Pro 1.4 & Desktop 10.5, keeping in mind that there isn’t many (or any) other 3D GIS you can do this kind of work in.

One of the 3D models built of Kendal, UK

Editing

Once you have converted or built your 3D model into a multipatch polygon you may find yourself struggling to edit or adjust your model.

Split

Having built entire cities, one issue that I’ve come across a fair bit is removing parts of multipatch polygons. For example, when adding multiple models, you may find that two features might overlap and need to remove part of one. In a 2D, standard GIS, you would simply turn the editing on and split the offending polygon.

This doesn’t work in a 3D GIS…Think about it, how does the GIS know where all the planes you require breaking lie with a 3D plane? 3D GIS can do some pretty clever assumptions but I am yet to find a way to remove part of a complex multipatch polygon.

True, within ArcGIS Pro 1.4 there are some editing tools for 3D multipatch features but it is early days at the moment. For simple cubes it is quite easy to adjust and manipulate the model a little but you don’t stand a chance if you have a curved edge.

 

Don’t despair, don’t give up on the 3D just yet, as you know, 3D isn’t new and there are many “workarounds”. One of my favourites is using the ArcGIS Pro “Replace Multipatch” tool. If you want to make multiple edits to a model (feature) you can export the multipatch to a collada format or keyhole markup language (kml) format, edit it in Sketchup, Blender, Meshlab or any of your favourite modelling suites and then import it again without affecting the other features in that layer.

 

If you are extruding simple 2D polygons with the intention of creating 3D mulitpatch polygons, it is a good idea to keep the conversion to multipatch until you explicitly need to. This way, you can edit, split and reshape your 2D polygon within normal ArcGIS Desktop without any issues at all and then draw up and extrude when you are 100% sure it all fits and works okay.

 

In my experience, pre-planning and clarity of the end goal means that you can be prepared for these slight niggles in advance.

Using 3D multipatches in ArcGIS Desktop

Overlay analysis 

So, I’ve built out an entire city of 3D buildings, it looks amazing, last thing to do is clip them by the city boundary….oh, I forgot, the 2D polygon doesn’t truly intersect the multipatch polygon in 3D space, the “clip” tool doesn’t work, the “intersect tool” nor the “merge” tool…in fact you can ignore using modelbuilder.

Clip features

I learned the hard way that 3D features work best with 3D features. Unless the boundary polygon is a 3D feature, then you won’t be able to use spatial analysis to use any kind of overlay querying.

But, there is always a way to trick the system…

Although the multipatch polygon feature is a 3D object, you can still open it in ArcGIS Desktop, meaning that you can use the good ol’ fashioned “select by location” tool. No, it shouldn’t work but it does, furthermore, you can then export your selected data to a new multipatch feature. It begs the question why you can do this and not the “clip” as, in geoprocessing terms, they are the same thing (the clip tool is just the separate bits combined into a single script).

Let’s not get too hung up on it, as it works,

Volumetric Analysis

If you thought that the  “volume” tool was just put there to look clever, think again. It can be a great tool for easily representing floor space or calculating the tonnage of aggregate that needs extracting from the ground to lay a new pipeline.

There is a slight problem though…it can be a little hit and miss, especially when using sourced models.

Calculating Volume

Try adding a model you built in Sketchup, Blender or Meshlab to ArcGIS Pro and then calculate volume (using the add z information tool), Nothing, right? But why? The reason is that it doesn’t like “open” multipatch 3D features, it isn’t fully enclosed. Even if there is a slither of a gap in the polygon and it isn’t 100% enclosed, the tool cannot calculate the volume.

There are other methods whereby you can use a raster surface, like the “surface volume” tool but this isn’t quite as accurate as using your super detailed vector multipatch.

You could try the “Enclose Multipatch” tool, as this closes the multipatch and then running the volume tool BUT you need to consider that unless the multipatch is cut to the surface, for example, where a building sits on a hill and the base isn’t perfectly flat, the volume will not be ideal. So please consider using the data as a high resolution TIN which is merged with the terrain to provide a more accurate volume result.

Oh, last point on this – make sure you use a projected coordinate system that uses metric for your data, a geographic coordinate system will leave you with your volume in degrees….is that even possible?!

Which brings me nicely to –

Issues occur when you don’t specify the datum

Vertical referencing

I distinctly remember my first adventures into 3D through Google Earth, trying to create some of Romsey, UK as 3D buildings using Sketchup. The first hurdle was always figuring out whether the building was “Absolute height”, “On the Ground” or “Relative to the ground”…I mean, what does that mean anyway? I just drew it to sit on the floor, why does it need to ask me more questions about it?

Correct Height Placement

Right now, if you are a regular reader of xyHt or a 3D Geoninja, you will be calling me a muppet. In reality though, you wouldn’t believe how often I am asked about this, especially now that Digital Surface Models (DSMs), Digital Terrain Models (DTMs), bathymetry and other elevation data are so readily available.

Elevation is never easy, worst still, there are either too many or too few options. With the Esri ArcGIS Pro, I am 100% confident that I know where my data sits within 3D space but it is only because I’ve worked with this day in and day out and understand the limitations and data sources.

Let’s consider the Esri “scene” – it’s a cool 3D map and as you zoom into that lovely globe you can see lovely mountains and valleys all popping out of the surface, my question to you is, what elevation data is it using? what is the resolution of that data? You see, I love that Esri provide a detailed and complete coverage elevation surface for the entire globe but the flip side of it is that you cannot know the exact limitations of that surface easily (the information is provided by Esri but it is not a simple “point & click” exercise).

My words of advice here are to use your own terrain when placing 3D multipatch features. Therefore you are in control of both the vertical datum and the resolution of the height.

While I’m here, I want to also point out that there isn’t a “snap to ground” feature in the editing tools within ArcGIS Pro either. This becomes an issue when you bring a model in which isn’t vertically reference, has no vertical datum because you then need to sit it on the surface. Even when your model is a captured point cloud and accurate to 0.5cm, you have no way to accurately place it on the ground. You can adjust it up and down and sit it by sight, though you cannot “snap” it.

The big takeaway here is that firstly, you need to ensure you are confident and know your elevation data if you plan to work in the 3D scene views and secondly that you need to set up your x,y & z coordinate systems correctly from the start to ensure that all the work you do is as precise as possible.

…and yes, I now know the difference between “absolute”, “relative to the ground” & “on the ground”….maybe an interesting blog for another day, though feel free to contact me if you need quicker answers!

And everything else

There are still many things I have not had a chance to mention, for example the complexities of cartographic representation using 3D models in a GIS, or ways of minimising the clashing of overlapping data plus other 3D centric issues such as shadow and light. Maybe a blog for another day?….

Dragons8mycat

 

 

QGIS 2.5D AGAIN….But on a web map!

You can now export your QGIS 2.5D maps straight to a web map thanks to the genius(es) Tom Chadwin & Luca Casagrande who developed the QGIS2Web plugin. Before I run away with myself and start talking 2.5D and cool effects, I think it’s best I clarify a few things.

dragons8mycatLeave a comment

No, you didn’t misread that, you can now export your QGIS 2.5D maps straight to a web map thanks to the genius(es) Tom Chadwin & Luca Casagrande who developed the QGIS2Web plugin. Before I run away with myself and start talking 2.5D and cool effects, I think it’s best I clarify a few things.

Firstly, I realise that I told a bit of a fib when I wrote the xyHt article on web mapping where I said that there wasn’t an easier way to build a web map. There is, it is QGIS2Web, the only thing you need is your own website and QGIS. You simply make your map and hit the plugin button….voila! Not only a preview of the web map but also options for measure, popups, scalebar and even basemaps – It is truly a thing of beauty. The only technical knowledge you need is how to copy and paste your folder onto your web host.

It sounds too good to be true, which is why I feel guilty for ignoring it for so long….Having been beaten by the geospatial industry for well over a decade, I naturally assumed it was some scam whereby I would have to buy into something or pay for a subscription but no, this is the real deal. This is what ArcGIS online should have been, you front the cost (or not – have another look at Github pages *ahem*) for your website and the rest is free. You can host as many maps as you want with whatever style and data….of course, with a little know how and you can even link them to other sources using hyperlinks in your fields.

Screenshot from the 2.5D map I made
Screenshot from the 2.5D map I made

So, let’s clarify what 2.5D is

2D is the everyday “flat” maps which you would generate in QGIS/ArcGIS/MapInfo/CadCorp (add your own here) although there is relation of how features lie in comparison to others, there is no depth. Buildings and trees appear as “top-down” flat objects.

In true Dragons8mycat style, the best way to describe this is using a 2D image of Super Mario Bros:

Super Mario Brothers in 2D
Super Mario Brothers in 2D

2.5D Adds depth, though it is not full 3D, it cannot be explored like Google Earth, rather it is an illusion of depth.

This can be seen in the image below of Mario in 2.5D

2.5D Super Mario Bros
2.5D Super Mario Bros

3D provides true depth and the ability to explore the depth, this would be the Google Earth buildings or the CityEngine webscenes. Unlike a fixed view of the 3D features, they allow full movement around the real-world feature. Here is Super Mario once again to show what 3D is:

Super Mario in all three dimensions
Super Mario in all three dimensions

Is this the right time to discuss 4D?….No?….Briefly then – 4D is the medium of time, if we were to add a timeslider to the 3D Super Mario above, and  we could move around not only the features but also the time, then it would be 4D (temporal).

Out of interest I have had many discussions over what 3.5D is and so far the general consensus is that it would be a 2.5D map with temporal capability, so think of a 2.5D map which showed change over time….could this (below) be 3.5D?

3.5D?
3.5D?

Back to the future (qgis2web)

The QGIS2Web plugin provides the ability to create 2.5D features on a web map, there is the option to adjust & change colour and you can add functionality. What really surprised me was how easy it was.

Of course there are some minor things which I found as I worked through but I am sure that these will be fixed before this post goes live as the developers of this plugin are right on the ball.

Tip 1 – Use geojson files.

Although the Plugin uses any QGIS data (WYSIWYG) I found the plugin quicker and gave a more accurate representation with geojson files

Tip 2 – Forget your shadows

I was told that the plugin DOES honour the shadow effects but I found in my experience that the shapeburst fills and shadow effects didn’t work and in some cases caused the map to hang when switched on

Tip 3 – You need to remove the OSM maps in the code.

There are a plethora of basmaps available, I’m not going to knock it, BUT there is no option to turn them all off or to have them off by default. If you want to remove them, find the layers.js file and then remove the baselayers reference. var layersList = [baselayers,lyr_LandParcelsSedbergh,lyr_Trees]; should be changed to var layersList = [lyr_LandParcelsSedbergh,lyr_Trees];

 

[iframe src=”http://dragons8mycat.net/QGIS2WEB/sedbergh2D/” width=”600″ height=”400″ scrolling=”yes” ]

 

Above is the map I made of Sedbergh, as you can see, the OSM baselayer is still on as I keep going through phases of liking it and then hating it.

I am sure you will agree that, once the shadowing and shapeburst fills work, it will be amazing.

If you need any further information or would like some training, please feel free to contact me.

Nick D

 

Further Adventures with 3D GIS

dragons8mycatLeave a comment

Originally Posted in xyHt on 19th November, the original article can be read here

Last year I posted about this awesome plugin called “QGIS2Threejs” [here] and how it was possible to quickly render 3D visualisations for free using the open source GIS, QGIS. At the time I fudged some wind turbines as it was impossible to import models, furthermore I avoided too much detail as any imagery would look poor when zoomed in. In short, it was good but was not an alternative to ESRIs’ CityEngine or a model you could build in Blender.

A year and a half later, it’s a different story [demo here]

OS3DThe wind turbine is a model I built in Sketchup and exported as a Collada file, the trees are created from Ordnance Surveys Open Map data (based on the woodland polygon), this is something which I could provide to a client to show how their issues interact with some confidence.

OS3D2Note in the above picture how the trees have a slight randomness, I really liked the way this came out using the QGIS vector tools.

Below is a breakdown of the data and steps to reproduce it.

Software Required:

  • QGIS 2.12
  • Sketchup or Blender

Data Required:

  • OS OpenMap
  • Environment Agency Lidar DTM & DSM

Steps to reproduce:

  1. Open QGIS and import the OS data, style either with the QLR styles provided by Ordnance Survey or Ross McDonald personally, I created my own based on the Ordnance Survey style as it is instantly recognisable.
  2. Using QGIS, bring in the DSM and intersect the height with the buildings to add a z value to the buildings. I found that adding this height as a field was good for checking later on. Make sure that you use a mean height for the buildings so that you dont end up with strange shaped roofs.
  3. Using only the woodland layer of the OS OpenMap data convert to random points. Remember that your “average” tree is about 5m high and you want about 2m seperation (radius). Regular PointsThe regular points tool can be found in the ‘Vector’ tab under ‘research tools’ – Don’t be tempted to use the ‘random points’ tool as it is quite tricky to set the density right for the varying polygon sizes. There is a tick box on the regular points so that some ‘randomness’ is generated.
  4. Create a new point shapefile, this will be used to provide a location for you turbine/model. Obviously, put the point(s) where you want your objects to be.
  5. Create or download your 3D objects. For this part, I made the turbine with Sketchup to the correct dimensions. The Sketchup 3D warehouse has a plethora of 3D models, cars, planes, a Back to the Future DeLorean….they can all be exported in Collada format from Sketchup, just ensure they are located correctly within the space – the model will render in reference to the point file you create, meaning that the point will be used as the models 0.0. You can also build models in Blender, AutoCAD or Threejs – As long as it is in Collada format, QGIS2Threejs can read it.
  6. Double up your trees! You need to have a trunk and a treetop, therefore, bring in the tree data again…..
  7. Open QGIS2Threejs, it should be under the ‘Web’ tab in QGIS, if you don’t have it there, go and add it as a plugin.
  8. Okay, now for the fun part. Whatever is currently shown in the dataframe, is what will be used as the imge overlaid on the terrain…so tidy it up and consider that the buildings, trees & model will be sat on top (Might be worth turning them off in the layers) QGIS2ThreejsSet your DEM to reference the DTM
  9. Here is where having 2 sets of trees comes in useful – set your first set of trees as cylinders (as above). Set a radius of around 0.5 (assuming you are using a metric projection). Then height to around 5-7m, I found that although the Institute of Architects recommend 7m, 6m worked well.
  10. For the second set of trees, set the oject type to sphere, radius of 2m and height the same as the trunk. DON’T forget to set the colour to green & the trunk to brown!
  11. Set the buildings to the ‘Extruded’ object type and set the Height (NOT THE Z COORDINATE) to the height field you generated, you can also use the object geometry, though I find the field method more reliable.
  12. Set a file/folder location and then sit back and enjoy!

But it’s no ESRI CityEngine…

True, this is not CityEngine, nor would it ever need to be. It is a FREE plugin for generating geographically correct 3D models…but what if I told you that you could add buildings with shape? There are companies like CyberCity3D who sell 3D buildings and cities, therefore you can bring in more accurate buildings….

This is only the beginning for QGIS2Threejs, it is built on the rather amazing Threejs. Looking under the hood of the code, it is quite easy to add fog, shadows and adjust the angle of light. It wouldn’t take much for someone with a little more know-how in javascript to add a control for light dependent on time of day.

One thing is for sure, this is a great little plugin, with a lot of potential.

Nick D

GIS Tips – Think about how you are going to present it

dragons8mycatLeave a comment

A map’s a map right? You get the data and you make everyone happy with a nice map….not always.

Chatting the other day to a colleague and they don’t like using 3D, the justification was that there isn’t any reason to, which (as you’ve already guessed) got me a little wound up. So, I thought I’d have a look around on the web and had a real moment of “why did they do it that way?”

At this point, I wish to point out that my way isn’t (always) the right way, there are many considerations to be made when finding the right media for your geospatial representation, mostly it comes down to what the client/customer is paying for but as GIS professionals we should be steering the end user from some of the monstrosities around and making the results easier to understand

Don’t just colour in the pictures (all the time)

I like to think that rather than a scientist, I am an artist. One day I hope to see my work on someone’s wall or for someone to ask me to create them a bespoke piece like one of my previous works –  and to a small extent we Geomagicians ARE artists, we may not always produce works of art but we have choice over our presentation medium and we can pick the right colour crayons.

Much like the sculptor, the oil painter or modern artist, we can choose, over we use 2D paper maps, 3D videos, web maps or something a little more interesting (3D printing anyone?!) and our choice of medium should be driven by providing the end user with the best way of understanding the mass of information collected.

I’m sure that I’m more Escher than Dali

Let’s run through a few ideas –

You’re asked to produce a map of a site layout for a new building. It’s for a presentation which will be seen by a variety of people across a project…..

Yes, a topographic map would look great. Assuming that everyone is map savvy, they can all see where everything is going…..BUT….what if you produced the layout as a little 3D video clip, where it rotated around the building being put in? Even if there isn’t much detail in it, it would provide EVERYONE, even the non map savvy to understand the context and relationship the features will have on the ground.

You’re asked to create an environmental map to show the issues on a particular site, what you didn’t realise was that there were 15 overlapping layers. This is going in a technical report and only has room for 1 page…..

How much hatching is there in the world? You could again, provide an intricate layered topographic map OR use an interactive PDF – why not use the “geoPDF” format and allow the end user to switch their layers on and off? Although cartographically you would still need to contemplate how the layers will display, you are not restricted to ensuring they are all visible on print.

Your boss wants to show on a presentation, the urban growth for a small village over the last 10 years…..

If you’re thinking that a nice Ordnance Survey based map with the buildings for each year in different colours, you might not get that promotion – why not either a) create a time slider map (available in QGIS & ArcGIS) or b) create a little video showing the change over time??

This is what I created when I was asked to show the UK wind turbine rounds (turn your speakers up):

You are asked to show how a shipwreck lies on the seabed as the bathymetry doesn’t match the photo in the book.

Why not extract the image from the book and georeference it over the top of the bathymetry? Then animate it as a 3D video so that the user(s) can see it from all angles? That’s what I did….this is one of the early images, the detailed images are licensed.

So why bother?

I’m not saying that you should give up paper maps and PDFs, what I’m questioning is our laziness as data visualisers. Sometimes we take the easy option rather than remember that we are the story makers or the answer providers….and what Jack Dangermond is missing is that we are responsible for TRANSLATING the story and information too.

Next time you get a request for a map through, instead of saying “yes” and churning out another identical map to the other 30,00+ you already have, take a minute to think about whether the end user is going to understand the results. Don’t assume that you know the end user, instead show the receptionist the map and ask if she/he can explain it or maybe show your partner….don’t assume that you can think like a non map savvy user!

For every unhappy or confused client, you will lose 10 more through word of mouth and bad press……for every happy client you will not only gain a great sense of happiness, but potentially another 3 clients.

Nick D

ArcGIS 3D Web Maps – CityEngine Web Scenes

dragons8mycatLeave a comment

How often have you put a visualisation together and had to consider data provision as your first priority or created a work of art which then needs to be bundled onto a harddrive or disk to send to a client? I am relatively new to 3D visualisation and I’ve had to completely rethink a few of my visualisation projects to allow for it to be provided to the client, instead of creating an interactive model, having to create a video, just so it is portable.

It seems that ESRI have the solution and, to be honest, it’s about time! 3D visualisation through the internet:

ESRI Campus

If you follow @ElliotHartley you will have already heard about how the new ESRI CityEngine is revolutionising visualisation and just how QUICKLY you can populate an entire town, one thing that I don’t think we are hearing enough of is how flexible it is and that you can share models through the web.

Just have a look at some of the examples:

ESRI Campus visualisation: Here

LA Metro visualisation: Here

Venice Look a like: Here

Its early days but I can see huge potential for the renewables and environmental industries for using this with their visualisations for EIA or for visualising noise assessment.

If the text in the ArcGIS 10.2 guides are correct, there will be provision to export Models built in ArcScene to CityEngine format and potentially to the 3D web scene format….exciting times.

When ESRI get it right, they REALLY get it right and in this case they have nailed it, ESRI must have invested a lot of money into this and will need to recoup some of this loss, so my only wish is that it is priced reasonably enough for the average user to enjoy and not restricted to the public sector.

Nick D