Project: Building virtual models of the world using picturesProject: Automating reconstruction to quickly create virtual copies of real cities
Building virtual models of the worlds using pictures
Virtual models of the real world are typically built by a specialized team of designers and developers. Consequently, this costs a lot of time, effort and money. Now, imagine being able to do away with this by just going outside, taking some pictures of the desired scene, uploading them to a computer and pressing a key.
Virtual models of real world scenes are commonly required for simulations, entertainment purposes, navigation and many others. Instead of having to model these by hand, many researchers are exploring ways to do this automatically from a variety of data sources like imagery and laser measurements. Our objective is to be able to exploit the vast amount of real world imagery that is available, like the panoramic images that are acquired in large volumes, or the many image databases that are on the internet, and create complete 3D models from that data.
Estimating the 3D structure of what we see around us is something our eyes and brain do so easily that we don't even realize it. For a computer, working from captured images this is a lot harder. The problems start with finding corresponding points between the images. Research has turned out that even the most state-of-the-art methods are often not able to find out which parts of two images are the same when the images of a scene are taken from quite different perspectives, yet this is a key requirement in order to be able to calculate depth. Our research has analyzed the points of failure for commonly used point matching techniques. Based on these finding we try to propose improved matching algorithms that are effectively computing corresponding points between images, even when the images are taken from quite different perspectives. The matched points then are easily converted into 3D points that describe the scene.
Developing new methods to calculate the 3-dimensional position of points in the scene is but the first step. A set of points is not yet a model that describes surfaces, joined together at the edges to form a closed whole. Many existing algorithms try to find these surfaces from the 3-D points only. In our research we plan to combine the points that were found with additional information contained in the images. By virtually slicing the images apart into segments along the edges of components in the scene, we identify the surfaces which are needed in the model. Exploiting the principle that ‘3 points define a plane', by combining the segments with the position of the points, it is possible to calculate where the surfaces of the objects are in the real world. Combining these surfaces then provides the virtual model of the real world environment, exactly what is so desired.
1.1 Automatic World Generation Based on Real Data
TNO Defence, Security and Safety
Frido Kuijper, TNO Defence, Security and Safety
Workpackage 1.1 Automatic world generation based on real data
Automating reconstruction to quickly create virtual copies of real cities
Emergency services, municipalities, and location games require a detailed and upto- date virtual copy of a real-world urban scene. Automating reconstruction can greatly reduce the time and cost of the process. New methods have been developed that can efficiently perform steps in recreating the world. Soon the whole world will be completely modeled in 3D.
To create a geometric 3D model of the real world, we need to reconstruct the scene from data sources such as images and laser range. The detail and complexity of the reconstructed model depend on the amount and quality of the data, while the time needed for reconstruction depends on the efficiency of the methods, especially when using larger data sets. This project is aimed at automatically reconstructing geometric models from laser range scans of large real-world urban scenes. These scans result in large clouds of points in 3D space. The methods should efficiently reconstruct a detailedand precise representation of the real scene from these points.
The reconstruction process can be split into different parts like identifying existing infinite surfaces that pass through many points, bounding the points in these surfaces, and filling the remaining gaps where data is missing. Urban scenes contain many simple surfaces, and for the typical scene about a quarter of the surfaces are rectangular. We developed a new method for efficiently bounding the appropriate surfaces using rectangles. After different surfaces are identified in the data, each surface is checked to see if it should be bounded by a rectangle. We have developed a method to determine whether the data in a surface is well covered, such that there are no large parts in the rectangle that are void of points. The Netherlands Forensic Institute (NFI) has provided a laser range data set, and our method efficiently identifies surfaces in this data set and provides the rectangles that correctly bound each surface. Once rectangular surfaces are identified and bounded, the remaining part of the scene may be searched for increasingly more complex shapes. Alternatively, the shapes bounding the surfaces may be based on the data distribution instead of predefined forms.
Surfaces may be bounded using predefined shapes like rectangles, triangles, L-shapes, etc. or their boundary shape may be determined from the local data distribution. We are currently developing a method for data-driven boundary creation. This method not only takes into account the data measured in one surface, but also the shapes of neighboring surfaces. The method should result in a shape that is easy to connect to neighboring surfaces, while behaving nicely in the presence of noise and missing data. Because of measurement problems like low data resolution and occlusion, some parts of the real-world scene may not be captured in the data. Stitching together the bounded surfaces that are present in the data may reveal these holes in the data. We will develop techniques that can identify missing parts in the data and fill these parts in a realistic way.
1.1 Automatic World Generation Based on Real Data
TNO Defense, Security and Safety
Van Kreveld et al. (2009). Identifying well-covered minimal bounding rectangles in 2D point data. Proc. 25th European Workshop on Computational Geometry, pp. 277-280.
Remco Veltkamp, Utrecht University