How to create 3D environment models from drone images

Welcome to this Lumion 9 tutorial about creating environment models from drone images to provide a context for a house or building design. As we have seen in Tutorial 4, it is possible to import large and detailed environment models which are created from drone images or laser scanning. In this tutorial, we will show you how to create such models using just a consumer-grade drone and photogrammetry software. Photogrammetry is a technique which creates 3D models with photographs as input, where, in the past, you needed to mark similar points of different photographs manually, with the more recent generation of photogrammetry tools, such manual marking is no longer necessary, provided that there is sufficient overlap between adjacent photographs. Typically, 60% or more. The number of images that can be processed has gone up considerably, to hundreds or even thousands of images taken from one area. Drones are helpful tools to capture these images. The latest generation of consumer drones is quite affordable, and they can capture images in high resolution and quality. For this tutorial, we used a DJI Mavick 2 Pro, which has a 20 megapixel Hasselblad camera. You can set the camera to capture an image automatically every two seconds. If you fly slow enough, you can make sure that the required overlap of 60% is achieved. The rules for flying drones differ per country. In many countries, it is allowed to fly above private property. In this tutorial, we will, therefore, fly a drone around the edges of a piece of property where an existing office building could be replaced by a new villa. We will fly the drone around the terrain several times, each time with a different camera angle and one time with a different height. For flying, we used the DJI Go app, which comes with the drone, on a 9″ Android tablet. Fly up to a height where you can see the facades of the houses across the street. Fly along the boundary of the property area while keeping the camera direction the same. Make sure that for each subsequent picture, the overlap is 60% or more. Fly the drone slowly, so there is no motion blur in the image. When you are finished flying the square along the perimeter of the area, turn the camera direction 90 degrees and fly the same path again. Once you’ve flown the path 4 times, turn the camera straight down and fly again. Then, move the drone up to the max height of 120 meters and fly the path again. When you were finished with that, lower the drone and turn it towards the building. Lower it until you cover the whole height of the building wall in the camera view. Then, fly the square again while keeping the drone facing the walls. This way, the photogrammetry software will be able to create a 3D model of the current building, which is good for showing in Lumion as well on a different layer, so you can toggle between the current and the future situation. When you’re finished, you flew the drone 7 times around the boundary of the terrain, each time with a different camera angle. When you are allowed to fly the drone outside the perimeter, you can capture the facades of the surrounding houses or buildings from different viewing angles, which will result in a larger usable 3D environment model. In this project, we ended up with 670 high-resolution photographs, which we can now feed into RealityCapture, the photogrammetry software we used for the 3D model reconstruction. There are many other tools on the market, but we achieved the best results in the shortest time and for the lowest price with RealityCapture. Copy all the photos from the drone to the PC hard disk. Start RealityCapture. Click on “Add folder,” and find the folder with the drone images. Now go to the Alignment ribbon and click on the left-most button, “Align Images.” This can take a while. RealityCapture is now finding all the points in each image that match with points in another image, and it builds up a 3D point cloud of such shared points. Ideally, RealityCapture finds such match points in every image. But sometimes, like in our case, several separate point clouds are created from different subsets of the images, which RealityCapture calls “Components.” It is possible to still align these separate components manually, but this can be tedious. When the main component, with the largest number of matched images, covers the desired area sufficiently, you might as well continue with only that component and delete the other ones. The next step is the 3D model reconstruction. Go to the third ribbon, “Reconstruction,” and click on the left-most button, “Normal Detail.” This process can take several hours, dependent on the number and resolution of the photograph and your PC specs. A fast processor is needed, and lots of memory, ideally 32GB or more. In our case, this process resulted in a model of 68.8 million triangles. This is way too much to handle for the subsequent steps, so we need to clean it up and reduce the triangle count. We first cut out the outer area. We can do this by using the lasso tool and draw a circle, selecting the area that we want to include in the 3D model reconstruction. Select “Invert,” and then click on “Filter Selection.” After a few seconds, a new model is created with the selected triangles left out. Don’t forget to save your project after each major operation. The previous model is still available from the left side of the user interface. You can still use it for other processing steps. The new model still has 66 million triangles. Let’s now get rid of all the small objects that are not connected to any other object and appear to be floating in space. These will not look nice in our visualization. Click on “Advanced,” then on “Select the largest connected component.” Then click again on “Invert,” and “Filter Selection.” To get a first visual impression of the model in Lumion and in your CAD software, let’s make a new version of the model with just 100,000 triangles. Click on the “Simplify Tool” button of the Reconstruction ribbon. It brings up some settings in the bottom-left part of the user interface. Set the type to “Absolute” and the target triangle count to 100,000. Click the “Simplify” button. In a few seconds, a new model is created with 100,000 triangles. You can inspect it by selecting the “Scene” ribbon. Then, click anywhere in the model to select it, and then click on “Solid,” and then “Sweet.” The sweet display will show the textured model after you carried out the texturing operation. Select the “Reconstruction” ribbon again and click on “Texture.” This can take several minutes. To export it, click on “Mesh.” In the “Save As Type” drop-down menu, select “FBX,” then “Save.” That brings up another dialog box with some settings. Keep the default values, which will result in an FBX file with a single 8K-by-8K resolution texture in PNG format. This file can be imported in Lumion and most CAD software tools. SketchUp, for example, standard does not support FBX files, but there are several third-party plugins for it. We used the Simlab FBX Import plugin. The 100k triangle model performs well in SketchUp, where you can use it to have a first look at the environment of your design. Rhino, Vectorworks and other design tools displayed it with a good performance. For Vectorworks, we needed to export the model in OBJ format. Rhino accepted both FBX and OBJ formats. Although we will use another version of the model for the final visualization, we can now already import the 100k-triangle model in Lumion, to have a first look at the areas where improvements are needed. Lock the model to prevent the color light-up when the mouse is over an area of the object. At a distance, the model looks pretty reasonable. But when we move closer to the ground level, it becomes obvious that the trees, cars, light poles and the bus stop don’t look very good. These need to be cut out of the model and replaced with equivalent objects from the Lumion library. The road also does not look very flat everywhere and needs some flattening work. To do that, we go back to RealityCapture. Since working on the 66 million triangle model is slow, it is best to reduce the model now to 1 million triangles using the “Simplify” tool of the “Reconstruction” tab. Set the desired triangle count to 1 million. Then, click on the “Simplify” button. After a few minutes, a new model is now created with one million triangles. You can delete some of the previous models to make the saving process faster. Now, click on “Clean” model. This will also fill up some of the holes. Also, try the “Close Holes” function. Now it’s time to texture our model. Click on the “Texture” button. This process can take several minutes. Let’s have a closer look at the result. Zoom in and out using the scroll wheel of the mouse. Pan left and right by clicking the left mouse button down and moving the mouse. Rotate the view by keeping the right mouse button down and shifting the mouse. Some parts look pretty nice, but trees shrubs and cars don’t and need to be cut out of the model to be replaced by objects from the Lumion library. Small objects can be cut out easily by looking top down at them. Select them with the lasso tool, and Filter Selection. You can add to this selection by pressing the control key down. The resulting hole can be closed using a “Close Hole” or “Clean Model” operation. Note that the texture mapping gets lost after a “Close Hole” or “Clean Model” operation, but the model can be textured again at any time. It just takes processing time, so you probably want to make a number of changes before executing the Texture function. Sometimes, you need to look top down to cut an object out, while for other unwanted objects you need to look from the side. Make sure that there are no objects behind the one you want to cut out. You can always deselect parts of a selected area by pressing the shift key while using the lasso selection tool. This cleaning process is the most labor intensive part of the workflow and take more than an hour. Another part to improve is the unevenness of the flat areas like roads. The Photogrammetry software does not always make flat areas really flat. You can flatten these in two ways: reduce the triangle count and smoothing. The “Simplify” operation has two options: “Absolute” and “Relative.” Earlier, we used “Absolute” to reduce the overall triangle count of the whole model. Now, let’s select an area of the road which does not appear to be flat, and select the “Relative” option. You can set the percentage of simplification, for example, to 30%. This means that the selected area will be reduced to 30% of its original triangle number. If this is not enough, you can carry out this operation another time. Then select the area again and click on the “Smoothing” tool. Keep the parameters in the left side of the user-interface unchanged and click the “Smooth” button. For this operation, it may be useful to look at the scene in “Solid” mode. Click on “scene” in the menu, then, click anywhere in the model. That brings up some options how the model is displayed. Click on “Solid” in the “Scene Render” section. Normally, you set this to “Sweet,” which shows the model colored or textured, but in “Solid” view, you can see the unevenness of the road much better. When you are finished cleaning up the model, delete all the interim models that were created in the process and save. As the objective of the visualization is to show an area with the existing situation with an office building and the future situation with a new villa, we need to cut out the existing building as well and fill the hole. To save this building, go back to the previous model. Select the building again. Invert the selection and filter out the environment. Now, re-texture the model of the environment without the office, and the office model, and export each one. Before we import these files into Lumion, there’s one more thing we need to do: create a satellite image on a surface object, to provide a more realistic looking distant view beyond the area we captured with the drone. In SketchUp, this is easy. Start a new project. Delete the human figure. Click on the small icon in the bottom-left corner. It brings up a dialog box that allows you to set the geographic location of your design. Click “Add location.” Type in the address and click on “Search.” Scroll out to capture a wider area. Click on “Select Region.” Move the corner points to the desired spot. Click on “Grab.” SketchUp creates now a plain square surface with a texture map on it of the selected area. Right-click on it to unlock it. Save it. It is now ready for importing in Lumion. In case the area around your design is not flat, you can use the Oob plugin to capture the satellite map plus a 3D terrain file and map the satellite image on the terrain. To show this, we will use another area which has some dunes. The Oob plugin needs to start with a georeferenced terrain. So start by creating the satellite image on a flat terrain, like we saw in a previous step. Click on the Oob icon. It brings up a dialog box. Select “Satellite View” under the dropdown button, “Build 3D terrain.” Select “High resolution Google data (beta)”. Oob will now create a 3D mesh of the area, based on Google height data, and map a Google satellite image on it. After all the tiles are created, hide the first flat satellite area. Select all terrain tiles and explode them. Then group them. Smooth the normals under the “Soften Edges” toolbar. Save the file and your 3D textured terrain is ready for importing into Lumion. In Lumion, import the terrain file from SketchUp and the cleaned up models from RealityCapture. Put each model on a different layer. Also, import your new house design and put it on a separate layer. Now you can start placing hedges, trees, plants, cars, streetlight poles and people, like you normally do in Lumion. If you don’t know the name of a tree or plant, there’s an app that helps you detect it: PictureThis. When you found a match, use the “Search” function in Lumion to find the matching object in the object library. Here, a bird’s eye view of the result. Here, the same from a street-level view. And the resulting renders and animation. Here are an overview of the hours we have spent on this project, and of the costs. This concludes the Lumion 9 tutorial about creating environment models from drone images, to provide a context for a house or building design.

Comments 31

Leave a Reply

Your email address will not be published. Required fields are marked *